• Liana “Li” Evans, VP, Head of Learning & Delivery Enablement at CX Studios
• Jenni Bruckman, VP of Customer Success and Strategic Partnerships at WeVo
Li Evans:
Welcome back to the CX to the end factor podcast from CX studios. I’m Lee Evans, and I’m here with Jenni Bruckman from WeVo. She is the VP of customer success and strategic partnerships. And we’re here to talk today about how UX testing. Affects the customer experience across the board. And it’s pretty much from end to end.
You want to test all your different engagement points and we’re looking at it from a perspective of, from customer experience with CX Studios, we’re looking at it, how it touches from the time that you’re doing awareness to actually using it in acquisition customer funnels and customer conversion funnels, as well as into customer loyalty.
So I wanted to kind of get your opinion from working with WeVo Jenny. how has the traditional UX testing and research methods, how are they failing customers at this point? How are they trying to adapt to this new world?
Jenni Bruckman:
Yeah. So I think what we see is, the more data we have available, the harder it is to make sense of what that data tells us and then what to do with that data.
And we’ve spent. So much time and effort really focused on, industry experts are digging at the, what the quant tells us, primarily. And so you’ve got all these various data sources and you’ve got data visualization techniques and all this fun stuff to help you make sense of that quant data, but it doesn’t do anything to fill in the gap on why.
And so I think we’re most UX experiences get hung up. Traditionally caught in this counterbalance of, I don’t understand why our data is performing the way it is. you know, pointing to the break points in our funnel, that it is, or the lack of engagement that we’re seeing or whatever it may be. And then to answer that why some of the traditional methods to uncover that are typically just very labor intensive and, very hard to do.
And so we’ve seen a massive uptick and, that demand for insights, really, really creeping onto the scene even stronger than we’ve seen before.
Li Evans:
So do you think it’s a lot of, analysis paralysis, like too much data just is, you know, there’s so much out there it’s hard to really figure out what, what works and what’s not working.
Jenni Bruckman:
Yeah. I mean, I think part of it is that, There was a, such a heavy focus. I come from the space of AB testing and optimization, and that is really about a 10, 15 year old specialty within digital organizations. And so what we would see is that it’s a very long tail curve for people to understand, oh, experimentation as a key driver in understanding what’s going to work and what doesn’t before we just make assumptions and release experiences that don’t align to customer expectation.
But then we also have this, this piece of, once that gets off the ground, now we have really sophisticated organizations doing that either in-house or, you know, with, with partner teams that help them Excel faster, but then they, they still struggle to make the impact and the momentum and the roadmap that they intended, early on, because they don’t have that why piece.
And they typically are now plugging that in. and I think we’re seeing a similar trend to what we saw in AB testing and optimization. 10 years ago. I think we’re seeing that in UX research and really customer insights as a whole, and that is made up. Qualitative and quantitative, datasets. And so I, I think, you know, it’s, analysis paralysis to some extent if we have more data than ever before, but it’s also just that it’s hard and messy and complicated data.
And, oftentimes exists in different silos and different parts of an organization that are owned by different teams and how teams structure that and access it between each other. and then the. That they have to access and understand those insights is, is just widely varied. and so you’ve got different level experts, different access to different data sets and, and very rarely do we see those crossover where they have access to understanding their holistic customer experience from the same.
Li Evans:
Yeah. So speaking of the other silos, we see silos just really inhibiting customer experience because there’s one team that does the marketing. There’s one team that does the email. There’s one team that does, that builds the website or another one that builds the app. There’s a whole other team that does that data analysis.
So do you see, how do you see the silos really inhibiting customer success in this.
Jenni Bruckman:
Yeah. And then we’ll take that one, one level further, which is now these experiences exist on multiple marketplaces that you don’t typically get to own in their entirety. Right? You may cross over into other point of purchase experiences or consumers are researching your brand and product or service in a multitude of places that you may or may not own.
So not only are you trying to connect these siloed, perspectives of the experience. Across brand and media and, you know, the site, and digital and, marketing, like all these different teams are coming together, but now you’ve got all these, these experiences that even those teams don’t own or touch, exponentially increasing where your, where your customers are finding you or exploring you.
so yes, is the short answer to that question. I think we see an enormously disconnected view of, of the customer experience. Once you try to unpack that and understand that. Connected across the channels.
Li Evans:
Yeah. And well, we’re focusing on, in, in our training is, is teaching our students in CX studios university that we are, we’re focusing that it goes across all these different engagement points.
It’s not just, Hey, What, what is that expectation when somebody comes to the site, you know, finding out that user experience, when someone comes to the site, it’s also, what’s that expectation. When someone sees the bus wrap around, you know, the bus rapping and clicks the QR code, and then they come in to your app and then maybe they don’t go from your app.
They go, what made me? They don’t want to watch a video on the app. Maybe they want to know. To your website, or maybe they’re going to be in your waiting room, right? What’s the stuff in the waiting room on the screens, like how does that affect the visit with, you know, with healthcare provider, all those different things are different touch points and different silos own those, and nobody talks to each other.
So how do you cross that user experience and make it right? You know, make it be a successful. And what’s that testing look like? How, how do we test those screens in the doctor’s office or those screens at the deli counter or in the, in the store against w how does that go across the whole experience? I mean, there’s, there’s so many silos that just, you know, that nobody or nobody’s thinking about connecting testing, and then how do they affect that, that.
Jenni Bruckman:
Yeah, absolutely. So I think that’s a great segue into, I’m happy to share a little bit of content around, you know, how do we really think about that? What we call the insights lab. So, for example, as we consider that there are all of these different cross points and channels in that experience that you just described Lee, and I think, the more data we get access to the harder it is to understand them and the harder feels, and the more overwhelming it feels.
And so what teams do is they tend to throw up their. And they say either I’m going to exist on one singular end of the spectrum or another, but trying to find something in the middle is, is too difficult. And certainly trying to find something that answers both quant and qual and can cross those channels from a tool set that’s accessible to all of their, their teams, whether they’re expert level users or not, is really difficult.
And so as we think about that, that spectr as I like to think of it as insights and accuracy, Within that you’ve got sort of this bottom left quadrant, which is where no, nobody wants to exist. This is where if you’re just making gut based decisions and you’re, you’re not being data-driven, you’re not being insightful.
You’re not taking into account that customer feedback, you exist in a world of nothing but bias. So certainly nobody wants to live here.
Li Evans:
I could call that the hippo, and I have to give a Avinash Kaushik kind of a, a props up for that, that term, the hippo, the highest paid person’s opinion in the room.
Jenni Bruckman:
Yeah. Yeah, exactly. This is, this is where the hippos preside, for sure. and so the alternatives to then being just, you know, gut-based is you could be really high on, on user research. So more traditional UX research. You need a very, broad and experienced UX research team typically to do this responsibly, because there are really critical things like bias and, and.
You know, solid methodology and framework that you need to ensure exists to capture reliable UX research insights. And so this is highly valuable when you have that expert. but it’s very labor intensive. It takes a lot of time to talk to a handful of people and then to synthesize through the insights you get from that, to understand directionally even what were your key takeaways.
And so the danger of that is it can become, dangerous land anecdotal, and you’re making multi-million dollar decisions based on the feedback of six people alone, because that took you so much time and energy. The inverse of that is more quantitative. So if that’s the qual you’ve, you’ve got the quant on the other end of the spectrum and this is more retrospective.
This is the world I come from, maybe testing and optimization. And really what exists here is highly accurate, right? To the hundredth decimal, you know, with statistical significance, which is going to be a better or worse experience to whatever power you want to look at that. And that’s great, but there’s a really high opportunity cost with running AB tests or running multiple experiences through an experimentation engine over and over without certainty that you’re pulling the right levers or the best levers that you have validation and, and, confirmation that they’re.
And so while it’s highly accurate here and, and, you know, it’s that inverse of the qual, it’s very, you know, absent of the, why you don’t understand the why at the consumer experience. So this is where we really focus on how do we make a more accessible solution that lets you. I understand I’m really high volume, so it’s not just a half dozen people, but it’s hundreds of people worth of qualitative feedback.
That’s reliable. And then how do you also benchmark and organize and, you know, reliably make it easy to synthesize the insights from that because you’re standardizing the way in which you’re uncovering those, those feedback points. And most importantly, across those channels, how do you do that in a way that doesn’t require a technical integration?
And I think one of the biggest benefits of that. You don’t need the live traffic. You don’t need the tech integration on your own site. That’s great. You don’t have to get hung up going through red tape forever. Everybody’s throwing a parade for that, but taking that one step further, you also can test and measure competitive experiences.
And I think that’s a huge void in today’s. Consideration of those journeys, like I mentioned, and like you were describing, imagine that you have a direct to consumer site that also crosses over into Amazon and best buy and all these other points of purchase, or imagine that you have a, you know, major coffee chain experience that has a digital, you know, ordering menu as well as an app, as well as a loyalty program, as well as a, you know, a site experience.
And so you have to be able to connect the dots across all those. And test that holistically.
Li Evans:
Yeah. And, and the advent of the internet of things, you know, like our watches, you know, like I, you know, like I like my experience with my bank because now my bank comes up on my watch while I’m running. Oh my gosh, what the heck happened?
Do you know? So those, those things, like how, how do you weave those into the conversation and the pandemic itself like this? I mean, we are so far. Further so fast then, like if that pandemic hadn’t happened, we wouldn’t be where we are right now. And I think that even comes into this testing, how do you, you know, click the brick, you know, how, how are those experience?
How do you test those experiences? And that’s, that’s a really, you know, the really. At more thinking in advance, thinking that you have to get into. And I think we’ve all just kind of helps get the sort through all of that on there. And so have you seen, like the pandemic has really changed?
Jenni Bruckman:
Yeah. I mean customer expectations.
We’re already at a pinnacle point before the pandemic of expecting our digital experiences to be more personalized and tailored and everything there. But personalization, even as a concept has changed so much, it used to be deliver the right headline to the right person when they land on your site. And if you were doing that, that was like exceptional now, to your point.
It’s I get frustrated if my. I phone doesn’t, pre-populate the two factor authentication code for me that came from the banking I’ve experienced that I signed into. So, you know, all of these multiple channels have to be one experience and a holistic consideration for that. And I think, the biggest shift in that, that we’ve seen is I think it was Nielsen Norman that recently released a statistic that the demand for this really illustrates the demand for.
At large brands and organizations being higher than ever. We are collectively going to need 100 million research professionals in the space by 2050, which by the way, astonishingly is not that far away. That is a huge number of how are we going to feed and fuel this demand for insights. And so I think paired with the ability.
For the super pros in UX research to be able to gather reliable qual and the super pros and, and analytics, to be able to gather and measure reliable quant, you have to have a tool set that’s democratized table across an organization. Yeah. And, and democratize double with trust. You have to trust its methodology.
You have to trust the framework and the way in which it gathers that data. And then helps you understand that data. And those are, those are big, hard things, and that’s certainly what we strive to sell for. And then on the other side of that, where your team so critically fits in is great. Now, what does that tell it?
Like, what do we do with that? How do we make this better? how do we improve this and continue to optimize it? And that’s really the law.
Li Evans:
Yeah. And what we’re focusing on with our training is that we’re, we’re, cross-training, it’s not just somebody that is just that dedicated person to the UI, to UX testing it’s that, they understand the concepts and maybe you might have.
Somebody that really likes to pick up in that, but they also understand this needs to be measured. So, and it needs to figure it out along the journey. So maybe it needs to be orchestrated. And how do we pop in some, you know, testing on our email templates? You know, we might need that. So understanding that the UX isn’t just for, so I think so many people get into that silo of, we need to do that UX testing just on our website or just.
Just on our application. Right. They don’t take it that step further. And that’s what we’re trying to, to bring into the, into the whole mix of things that it goes further it’s end to end. It’s not, you know, it’s, it’s about even my loyalty program. Like how does, how does my loyalty program, is it easy to use?
Can I get my points quickly? Do I have to have. Long receipts. Like you get at CVS people make a costume out of, out of this. Like so many people make fun of that, right.
Li Evans:
So like that’s what, that’s what we’re working to try and do. because you know, just having somebody understand how to use, an AB testing tool, if that’s all they can do.
They don’t see outside and they don’t understand the repercussions. So that’s why, w what we’re trying to do is so important is, is to go into that, that test day. and I, and I think the pandemic has a lot to do with that as well, because that, that frame of mind, and that thinking has always been. One person one tool.
Well, why can’t it be one person across a lot of tools to understand? There are a lot of concepts to understand how important a tool like WeVo can be to every stage of that customer.
Jenni Bruckman:
Right. And, and having that ability to democratize access to insights in a really reliable and scalable way, you know, making it rapid and repeatable is critical to allow, you know, your team and, and marketers and UX researchers, all of these different roles have to have that, that access to not just insights, but insights that they.
That give them a lens of both the quant and the qual. And I think one of the big trends we’re really seeing, and it’s going to catch up with many organizations sooner than they’d like to believe is the combination of quantum qual. They are inseparable now. And I think many orgs are still a little bit lagging on that.
They don’t yet understand they’ve really sophisticated one or the other. They’ve really matured their programmer on one or the other. And they’re just catching up with the one that’s missing. And I think thinking back to that insights and accuracy specs, You can’t just have one or the other, it, you have to have practitioners who can gather, you know, and understand really the insights across both.
What is the qual telling you? What is the quant telling you? Because it’s about the what and the why. and I think what we see over and over again, it blows my mind that the industry win rate for AB testing is still around 25%. And then. Tells us, how often are, you know, going back to the idea of hippos, how often our intuition is wrong, if we all placed a bet on that, but also how often we sit there and go that test idea should have one.
I don’t understand why it didn’t. That’s a lot of time and energy spent to go on a, a sure thing or a bet like that. You know, the, the counter protective, mechanism for that is to say, well, let’s go do some really solid quant and qual problem discovery, and really ensure that we’re building a prioritized roadmap based around what we can validate is a real problem for our users.
And then we can start to validate some solutions and build a really prioritized and thoughtful roadmap that’s built on a more solid count.
Li Evans:
So kind of going to that, how do you, how have you seen the artificial intelligence and machine learning really affect this particular part of, of the industry? You know, like from UX testing perspective, AB experience serving up the right experiences, how do you see.
How, how has that affect it as a positive, negative, you know, overwhelming.
Jenni Bruckman:
Yeah. I mean, I think, it’s, it’s for the most part positive, too skewed, very fearful. I think it was some stat recently that I heard, from pew research, I want to say that was 72% of Americans are worried about a future in which robots and computers are going to replace.
Purpose and that’s a valid concern, but I think what we’ve understood time and time again, is it’s part art, part science. And we can do a lot of heavy lifting with AI and make our processes and our, you know, reliability and our standardization and things like that. Much more scalable and much quicker and more accessible to do over and over again.
But we still need humans to provide the strategy and to design the test idea and to, you know, study and know the UX research that, that needs to factor into designing it. Well, and then alternatively, to, to say, here’s how we’re going to thoughtfully choose which pages in which order we want to test to understand what be.
Right. And so there’s, there’s still, we call it human, augmented AI. There’s there’s always going to be a balance of that. And I think, our CEO at Livo said it best. He said, synergy not separation is going to be the key to success. It’s that these things cannot be separated. but when they work together, it’s a heck of a lot more powerful than when we resist one versus the other.
Li Evans:
Just gives more trust into it because you still have that, that human eye and the empathy, you know, like you need, and that’s a cornerstone of, of customer experiences that empathy. And if you go go in with that empathy, there’s no way like the testing, the testing is just cold, you know, you can’t, you can’t put context to it.
And empathy helps to, to do that, especially in, in customer experience. so like going from that, that understanding that. Empathy. There’s not a lot of people out there that have that, that understand that in this CX, you know, in the CX world, a lot of people think it’s, oh, it’s just this one tool.
Right. And I, I know how to use this tool, but there’s a customer out there. Why are they rage clicking? You know, you got to understand why are they rage clicking, right. Because it doesn’t, it doesn’t work. And finding that talent is just, it’s really tough. That’s why there’s a war out, out there right now.
You know, like this it’s one of the biggest things that CTOs and CEOs and even CMOs are looking at right now that there’s not as many. There’s not that talent out there. And it’s tough for companies. Right. To change that mindset. The typical agency out there is very focused on a project or implementing a tool and they have a person to do that, but they don’t look beyond that.
The customer themselves needs people who understand across the board, that whole CX piece, and especially into the UX, you know, going back to your, your, your factor before about, you know, needing that hundred million. Research professionals, like, where are you, where are we going to find that talent?
That’s that’s just not going to be there. And so how do you build teams across? And, and that’s what we’re focused on. Like building these multi talented, multi solutionists that understand across the board to help rectify some of this, you know, this loss that this, this talent war, because of. The agencies themselves, as I was starting to say the agencies themselves, for them to make this change, to be able to, create multi solutionists.
They have to take people off the bench and they have to train and that’s hours upon hours upon hours. And what we’re investing is we’re making the back that we can train people to understand how do I leverage Wevo in the UX of not just. I’m building a site, but how does it go across the journey?
That’s what we’re betting on is that this, these, this training is going to, enable us to build a crew, a giant crew of, multi solutionists. So, so that’s, you know, when I get, I get excited when I get to teach people about Wevo and like, Hey, you don’t understand, like, it’s not. It’s so much more.
Jenni Bruckman:
Yeah, and I think, I think about it as you described that a visualized, you know, the, and this is going to date me, but the old trivial pursuit pieces, that is probably all digital now and by robots, I don’t know, but the old trivial pursuit pieces and you had to get all the different, you know, six different colored slices of the pie.
Yeah. And I think that’s very akin to what you’re describing is when you can fill in a really. Foundational understanding of expertise and skill set of expertise across, a really well-rounded. you know, understanding of the industry needs as a whole, that’s a much more valuable team player that you can plug in and scale to accomplish so much more outfit, I think, to your earlier point of analysis paralysis.
People withhold progress or perfection way too often. And so that also applies to staffing and teams. I mean, the, one of the biggest things that we hear about WeVo now is, is one of our biggest, you know, value propositions to our customers that we never would have predicted three or four years ago, which is I can’t hire people fast enough to keep up with.
Insights that I need to understand about my experiences to improve them and optimize them on an ongoing basis. I can’t find people to begin with let alone the pace at which I know I need more of them. And so, you know, we’re, we’re really, so well aligned to your vision of, and I just love it. I love the idea of like, if you have these brilliant utility players who can do all those who can fill in all those parts of the trivial pursuit wheel, They are so much more enormously valuable to accomplish, you know, a much greater scale of, of output and.
Li Evans:
Yeah. And like so many times I’ve been called like, Hey, you’re a unicorn, like recruiters have come after you. Because when you understand across the marketing stack, all these different tools, it, it really sets you apart because not many people will, will cross, especially if they’re there in their, their bubble of, Hey, I know how to program for a content management system, or I know how to do, you know, data analysis, right.
But they don’t step out. And, and that’s, that’s a, that’s a product of the agency world at this point. And that’s, that’s one of the things we’re betting on is that we can get people that want to do a change in their career or they’re just out of school and they want to try something, you know, We look at people from all different types of backgrounds and help to bring them in and give them the foundations that they need and in programming.
But then we come in and we, we were doing a whole thing around. UX, you know, like we’re, we’re focusing on UX tools so they can understand that again, I keep going back to it, but it’s not just the website. So many people just have that, that focus on that silo of thinking of it’s just that. And there’s, there’s so much more you can do.
And WeVo also can, you know, cause it isn’t just about a screen. It’s it’s about an image it’s about. You know, different things and when you can start to figure out, Hey, my content, if you pair it with content affinity, then you start to get really a lot more powerful that you’re working with.
Jenni Bruckman:
And to your point, you have to look at these different lenses.
It’s not just the design and the UX and the messaging and the copy and the images. It’s also the value proposition of the product or the service and the brand credibility and all these different layers are part of the holistic. The site, the page, the journey across multiple touch points in that end-to-end experience.
And so I think you’re, you’re spot on and, and creating a stable of, really a well-diversified portfolio at the macro level of a bench that can go in a multitude of directions. Responsibly and really efficiently, deep versus just a limited number of people who can go in a limited number of directions really deeply.
it’s such a better vision for, you know, you’re creating a center of excellence of people. And then each of those people is their own center of excellence, which is a really cool vision. I can’t wait to see it all come to fruition.
Li Evans:
I can’t wait to get my students graduated through here. It’s going to be a lot of fun.
What, what I want to also cover before we wrap up here is I want to understand some of the successes that you guys have had with, with some of your clients, using Wevo.
Jenni Bruckman:
Yeah, absolutely. So I think, a lot of what we hear is, especially in the, pandemic season has been not only how do I understand and improve the insights about my own experience, but how do I take a broader view to looking at my competitively?
more responsibly. And so, one of the biggest requests that we’ve gotten over and over again is for, brands or companies to look at their own experience compared to competitive experiences on a regular basis and looking at not just how. My experience perform. And what, what was the why? And, you know, what was the quant and what was the qual, but also, you know, how did that perform against industry benchmark and how are the industry benchmarks changing and how are my customers experiences changing and how are my competitors experiences changing?
And so I think, one of the biggest, demand areas that we’ve really seen grow is in that space of looking across, a view of not just your own experience, but that competitive marketplace. Tells you so many different, you know, we, speaking of silos, like we get, it’s hard enough to do our own experiences, our own brands and, and, products and services as well, let alone to find the time and energy to go examine others.
And I think that’s one of the most popular things we’ve heard is you can just go test any of those additional experiences and not have to have access to their site or have their profits.
Li Evans:
That’s pretty. That’s a very, very awesome, awesome, awesome aspect that I love about WeVo that, you know, Hey, I want to understand why my competitor over there is doing so well.
What’s, what’s the appeal, you know, and if I had something similar, would, would that work and it’s not just the hunch anymore. You have the data to back it up, You can test it.
Jenni Bruckman:
Absolutely. And I, you know, that one of the, that taking that all the way down the field, we’ve had one customer came up with the request and then it’s just taken off like wildfire, which is, can I just look at my competitors on a quarterly basis?
And I don’t have to think about it. You just go run those tests for me, come back and then share those insights. And now I’m able to keep a really close eye on that marketplace while I’m also keeping an intentionally close eye on my own experiences. So, yeah, that’s, that’s one of the big things. And then I think to your earlier point, just really stitching and connecting across journeys, has been the other big growth area that we’ve heard.
It just huge demand on. So going from app to, you know, site or, or menu digital menu and a drive-through for example, Going from email loyalty, right? Going from a loyalty email to the landing page. That’s one of the most disconnected experiences over and over again. And there’s no incentive. It’s, it’s completely devoid of to your point on empathy.
What, what real users care about what we care about when we’re shopping for.
Li Evans:
Well, I want to thank you Jenni, for your time today. This is a great conversation. so thank you all for sticking up, sticking out, out with us. If you’ve you’ve listened, we’ve had a great conversation. thank you, Jenni Bruckman from WeVo and I’m Li Evans from CX Studios.
We will talk to you next time.
Jenni Bruckman:
Thanks so much for having me Li. It was great.
