I'm pretty excited for today's episode. I might say that every episode, but it's genuine. I am very excited for each episode, but especially excited for this one because we're trying something new today. So we're kind of going to lean into a little bit of a series approach here, and this series in particular, we're going to call lab experiments.
And within each of these lab experiment episodes, you're going to hear from a member of the refined lab team, maybe a couple of them. Today it'll be Ashley talking about an experiment that she is currently running for one of her clients. We'll do our best to kind of, you know, make sure that we're not calling out any specific brands or names or technologies, and really kind of sticking to just the high level experiment, some of our hypotheses, some of the results that we're seeing, and yeah, kind of sharing some of the details so that our followers, our audience, is able to, you know, maybe hear enough that they could try similar experiments, or at least, you know, have a little bit more information as they're thinking through some different strategies for their upcoming marketing initiatives. So with that, we can kind of get into the meat of the discussion today.
And I think really at the highest level, this particular experiment is all about messaging, positioning, and how those two things, which are oftentimes kind of conjoined, how they can really make an impact on the user experience on your website, and ideally the conversion rate of people that are landing on your site and clicking through, some of the demo requests, so on and so forth. So with that, let's get into it. So Ashley, tell me a little bit about, I think the best place to start is like, you know, what's the problem you identified, or the challenge that was brought to you, the question that you were looking to answer. Tell us a little bit about, you know, where you started with all this and whatever background you think might be necessary.
Yeah, absolutely. So first, I always question messaging. I think it's something that is constantly needs to be on the docket of things to evaluate. And so this whole experiment came about of looking at the website specifically, so I'll pick on the website where this experiment, but it can be, we can go into how it'll be expanded upon after that.
But on your website, are you clearly communicating who you are, what you do, and why a customer should choose you. So oftentimes in marketing, especially in B2B, SaaS, it often can get very fluffy and very marketing heavy, like the top rated, or all these kind of buzzwords. But when you take a step back, a lot of times your audience doesn't even know what that was. You know, I, in a previous life, had a competitor redo their entire site, and I'm like, I don't even know what that means.
And I'm in the space, you know, so how do you feel that your potential customers will feel if they read that messaging? So often it's kind of like getting out of your own way as a marketer and really evaluating that is your messaging hitting, and is it clear, and is it concise? Because that will always win over being quote unquote, clever. So that was the problem that we were looking to solve.
We were looking at the website as a whole, because that is kind of like your home base, you know, it's where people land, it's essentially your home where everything sells. It's your sales team that's always on as a website. So we looked at and we evaluated, is the site performing well? And so we looked at different site metrics and Google Analytics, like bounce rate, average time on site, pages, recession, and then conversion rate, of course.
And so we looked at a lot of those different things and then evaluated and looked at the copy and asked, is this resonating? And so the hypothesis was with testing the messaging, can we improve the site performance just on that alone? So that's where the experiment really kicked off from. So that's super interesting.
So it almost seems like this was a little bit more of a proactive experiment than then reactive. Is that is that a good assessment? Like were you seeing when you were digging into some of the the GA data, was there anything that jumped out at you that said, like, oh, boy, like, this is a really low performing website, or is this more of just kind of like, you know, hey, this is a great practice for marketers, always be questioning things, especially that that homepage real estate that's getting so many views. So yeah, like kind of, yeah, help me understand how this how this came about.
You're spot on it was a proactive experiment. So it was really just questioning the, you know, our thought process and getting ahead of and looking for additive ways to drive growth. Because if you really think about it, I mean, if you can increase your conversion rate just by like 1%, which is very minimal when you really think about it, you get the same amount of traffic, but with that 1% higher, it could yield 100 plus depending on what your volume is and your current conversion rate, 100 plus more conversions. I mean, that's money right there, you know, that you could be making or leaving on the table if you're not constantly being iterative with your site as well.
This is a mistake that I see with a lot of marketers. Oftentimes there's this huge website project, whether it's a brand refresh or a site relaunch or whatever it may be, where you spend all this time creating this beautiful site, you launch it and it's almost like a site and forget it. Like we're not going to touch those four six plus months. I actually think that your website should be something that is talked about almost daily.
You should have an experimental roadmap, you know, outlined out so that you're constantly testing so that you're using the time that is passing anyways to figure out what's working or what's not. So you can use those lessons beyond just the website and it can carry out, not through just marketing, but your product team, like how they're talking about the product or maybe what they're building, even into sales. It just goes the whole game of the entire organization. So I think it's just something really important that marketers should constantly be working on.
That's awesome. So, so yeah, let's get into some of the granular details about, you know, about how you go about kind of testing some of the messaging on the website. So did you approach this and almost as like a split test, you know, where you've got kind of a different experience for an equal amount of people that are landing on the site, or was this just a, you know, we know what our baselines are and so let's change everything and see how it performs. So, yeah, give me a little more context there.
How did everything kind of come about and, yeah, what was the process that you followed? Absolutely. So we did use a technology partner in order to do a little bit of the preliminary research. You do not need to use a technology partner, though.
I can give different free ways to go about this as well. But we did use a technology partner that basically took their target audience and it pulled them. They asked for the qualitative feedback of what do you feel or what do you think when you see this copy on the site? They gave really, really great feedback with a couple of iteration rounds and then they launched that new copy on the site.
So we took the qualitative feedback from in market prospects. Then we put it on the site. It was a baseline, so it wasn't a clean A.B. test.
I will call that out right away. It was not a clean A.B. test. I typically recommend a clean A.B.
test just because it helps eliminate a lot of the what if questions that you may receive after it. Like, well, is it because of seasonality? Is it because of a certain campaign that was like whatever it may be? So I will call that out.
Should be an A.B. test. You can use different programs in order to do that. And you cleanly split it.
It just allows for a little bit more of a fair playing ground. But what we did was here's the baseline of what it was prior. And then we launched on X8, now what's the performance moving forward. And can we compare that to the previous period and see what the performance looks like?
So that was more of the like how it was rolled out. But yeah, again, here to do this, recommend the clean A.B. test. Yeah, I like that.
That's a great tip. And so I did get a little bit of a preview around what the experiment looked like that you shared with me in advance of this podcast recording. And so there's something really interesting that I noticed, but I'm not going to give it away yet. But tell me a little bit about like what you were, I guess in terms of like the messaging on the website, how did you decide what you wanted to test there?
Right? So, was it like your hero image and copy? Was it something a little bit closer to the fold or even below it? Was it everything?
Yeah, tell me and provide a little bit more of that detail for some of the audience here who's just listening and doesn't have the visual aid that I did and kind of walk us through as best as you're able to, you know, what the actual copy test looked like. You got it. I'll try and paint it as best as I can. So what we tested first was the hero copy that lives above the fold.
I highly recommend tackling that one first. When you look at the psychology of where your eye goes, just with you, you know, UX design, you go to that hero text first, then you'll read it right underneath of it. Typically when a lot of people are evaluating X8, they instantly go to the design, you know, we're going to test X It's actually more impactful to test your messaging first. Usually messaging has a higher impact when experimenting with, so we went with the copy.
So, experiment with the hero text first, and then also what went below it. One other thing that changed was the ZTA button, which I would recommend keeping consistent just as, you know, to eliminate one of the variables that you could say, well, it's because we switched to this. So that is another thing that there's never a perfect A-B test. I will be the first one to admit that.
That is something that learning lesson, you know, if you're going to be outlining your own experimentation test, make sure you're not changing too many variables at once. So we did the hero text and then underneath of it. And where this came about was the feedback that we got from the prospecting audience was, hey, I don't know what you do, or why are you different than your competitors. And so it was very like clever, like copywriting that was, you know, a spin on words and adjectives and things like that.
That was the hero text before to make it feel very creative and things like that. And it's a little bit heartbreaking as a marketer to hear that doesn't land. Because you're like, oh, I spent all that time working on this. It was so dang good.
But I mean, it's valuable. Good copywriting is copywriting that converts. So experiment with that, and then, you know, clearly called the main message that we want to get across in the hero. And then bullet points that go along with like those productive generators were what sets it apart underneath of it.
And so tested the homepage first. And this was all about the fold. It went through a couple of iterations of asking, you know, for feedback on this, you don't need to do it where it's in a bucket of this, you know, feedback ground, you can also do it in other ways too. I think, you know, the process of relatively stays the same, whether you're testing it, you know, by launching and then ask, you know, letting the results roll in and see if it works.
So we started with the homepage, let that one go. And then I think you should always start with the most trafficked traffic page on the site. And then you move from there and you start to spider web out. So next game, the demo page, because of course, that's one of the most like highest impact pages and one that you for sure should constantly be testing on because if you can squeeze out just a couple more, you know, submissions there, again, money.
So then went into the demo side. Same thing. So only experimented with the hero. So it's a two, two fold.
So copy on the left side, form on the right, all about the fold. So the hero text and the sub copy underneath, which went into bullet points, that was all changed as well. So it kind of, it was a cool process to see, starts with one side. Now we're going to go move in and do this process on another page and just kind of keeping that flow going and really focusing on just the most impactful pieces of the site.
Yeah, that's, and that's, that's great detail. I think for all those out there that might be more visual learners. Yeah, I don't know that actually could have painted a better picture than that just by talking through things here. But I think I want to get into hearing about the results of the experiment.
But I'll tell you what was so interesting to me about this from what you would share with me ahead of time is that from the looks of it, the higher performing version here was the one that actually had more copy and more context around it. And so often in marketing, right, we're taught less is more, you need to be more concise. Everything needs to focus on, you know, you, you, you messaging, what's in it for you. But what looked like one out here was the copy that had more context, like you were hitting on before, it hit on the product differentiators, which again, goes against a lot of the common rules that we hear, which is, you know, focus on the benefit to the user and don't talk about yourself.
But meanwhile, the data was kind of suggesting otherwise. So, so yeah, let's get into the results. Tell us a little bit about what the data said and what some of those key takeaways were. Yeah, before I get into the results, I do want to go on like just a small rant there, because I think it's super important is I feel like as marketers were constantly told what's best practice, whether it's form length or some things like that, I think you always need to be testing assumptions.
So just because there's a standard thought that less fields on a form make it better, that's not always the case. And you need to constantly be testing those assumptions. They're great to be a baseline. I know like, you know, when you're doing, you know, site content and things like that, you were taught less is more, but that's not always the case.
And you really have to test those assumptions, because it can vary from company to industry to persona, whatever it may be. Some personas want more like they are very more consumption focused. So pitch and pulling yourself in a box of best practices can really hurt your performance. So that's my little soapbox before I get into the video.
Before you get down from the soapbox, I want to double down on that one. I've always thought that best practices are just the same shit that everyone else is doing. And if our job as marketers is to find a way to stand out from the rest of the field, why on earth should we all be following best practices all the time? Instead, you know, test, let the data tell you the right direction to go and throw all the best practices out there for most of the best practices out the window.
So okay, you get set down from your soapbox now and let's get into the results. All right. So results wise. So this is still very early.
So they're preliminary results. I will also caveat with that. So like in the first couple of weeks, we've noticed like almost a 40% increase in just the conversion rate. Traffic was a little bit lower, but that can be a multitude of reasons.
And that's not something that draws me back actually. That's very interesting, because even with less traffic, we're still getting such a high conversion rate. Nothing else marketing wise changed drastically. So it's not like we try to hold a new way of advertising or anything like that that may have increased the site performance.
But overall, about 40% lift on both pages, you know, give and take a couple of percentages as well. So I thought that was really cool and enlightening to see. And I'm very excited to follow along with it and to see how it shakes out with even more data until it reaches that statistical significance phase. That's really exciting.
So what would you say outside of, you know, make sure you're testing everything? What would you say your key takeaway was from this experiment? And more importantly, like how are you going to, how are you going to action that those insights? How are you going to continue to utilize what you've learned from the experiment here?
Yeah, one of the biggest takeaways that I have is that your messaging should constantly be tested and you should be leaning on not just what feels great internally. You should be testing in your market of what's resonating because as a marketer, you may say, like, this is, this is so clear, this is spot on, whatever, maybe, and that means nothing if your market doesn't agree with it. So I think you really have to like detach yourself from the marketing as well, and really let your market tell you what it looks like. So that's a huge one is that your messaging should constantly be this iterative approach and something that you're constantly challenging.
The second one is that your website should constantly have a CRR roadmap. And I'm not just talking about something like change the button color, I mean, very large impactful pieces. I like to break the rules a little bit with CRR because CRO typically can be very like restrained and you have to like follow this very detailed like process and I felt like it was almost like a barrier in the past when I worked in house. I'm like, I can't test anything because it's in this phase and I have to wait six more weeks before it reaches statistical significance or whatever, maybe, but by the time that experiment may have ran, what's the results is that impactful or whatever.
So being able to also break the rules with CRO I think is very important, but making sure that you're constantly having something on the roadmap and that you're constantly testing things on your site as well. And it can be multiple pages as well. Like you could be doing on the home page and one on your pricing page or something like that. The lessons that you're going to take from it are super impactful, not just for the marketing team, but as the product or the company as a whole, even sales to understand, hey, we were testing this messaging prior.
This is a messaging that's really hitting. So when I do outbound or when I'm doing any type of discovery calls or whatever it may be, this is something that lands with the audience and so I'm going to lean into that versus other areas I could have been testing before. So CRO roadmap constantly be testing. I think those are the huge ones and then how do you action this?
What are you going to do with these insights? I think it's another huge takeaway. So cool, we got these results. What are we going to do with this?
I think that's a cool conversation to get into. I really like the idea of saying, hey, this messaging really resonated over here. Which other channels can we let it flow into and see if it also resonates over there? Yeah, I think that that that to me is particularly interesting.
Would the same messaging that work for them on the home page on their demo page could you utilize that within paid media where the intention, the focus of the audience is much different. They might not necessarily know who you are, what your brand is, what you do at that point. And so would some of that, this is us, this is how we're different, would that same messaging that worked well and led to lift on the website, would that work within some advertising and paid media? So I think if you don't already have some plans to continue the testing, maybe you've got a next step right there because I think that would be particularly interesting as well to see just how far you can go with some of this messaging that you and the team seemingly uncovered there.
And one more takeaway that I want to offer just in terms of testing in general in marketing, something I like to say quite a bit is, you know, micro tests are going to lead to micro results. And I think, you know, if you're going to, like you said, test a button color, you're not going to get a whole lot of of lift from it. It's certainly going to take you a while to get to that statistical significance level. But more than anything, you know, you're not going to get any massive results that are going to come from a really minor test like that.
Unless you're, you know, a company that's just generating like millions and millions of visitors all the time. And, you know, you do have the data set available to, you know, to make those really, really small tweaks and, you know, see them play out almost in real time. But for the rest of us, you know, if you're going to test something, take some big swings, take some, you know, take some gambles and see what happens. And, you know, hey, you might be surprised at what the results tell you.
So this is awesome, Ashley. Any closing words or tips or, you know, kind of a preview of what you're thinking about next that you want to share before we wrap up here? Yeah, actually have one takeaway that I always recommend. I want to make this accessible to every single company and team size and budget size as something that you could do in order to get this, you know, feedback from not just your customers.
I know we always talk about calling your customers actually. I think it's a little bit of the opposite when you're testing website messaging. Ideally, it would be someone who's not super familiar with your brand or your site or something like that. Something I love to do is to ask someone to jump on a five minute zoom call where you share your screen and you show up, you pull up your website and you ask them questions while they're like consuming your site of things like, who do you think we are?
What do you think we do? What are your immediate thoughts? Where would you go in order to do this? How would you complete x thing like purchase or sign up or whatever it may be and watching them navigate your site and think out loud is super insightful and it's really, really powerful to get that type of feedback.
You can get this with technology, of course, but again, I want to make this as accessible as possible to every single marketer out there is it doesn't require a ton of money in order to get these type of insights. You can do it very creatively and even have capturing this and documenting it and showing it to the entire team to kind of unlock some brainstorms is super cool as well. Like you can have some really great insights that you come out with as a team. But I would say definitely lean into looking at your market as your testing framework.
So, you know, you doesn't have to be just you in your office coming up with what the new next copy is and just throwing it out there and seeing if the market will adapt to it, you can, you know, test it in other creative ways like jump on a same call or something like that. So, that's a huge one. The other one is just the cascading effective experimentation as well. So, next up with this one is start with the home page, move into other high intent pages and then go into ads next.
So, that's another place that if you can validate the message before you spend 20k plus on a campaign, you know, and see if it's going to flop right away like the messaging doesn't hit, you're going to save so much money, it's going to be a much better, you know, conversion rate, your campaigns are going to constantly get better things like that. So, I think you need to also have like a cascading effect of your testing framework and also apply the lessons from each one to stack some growth there. That's awesome. Those are great tips and the last one I'll add to it is document everything because you never know when you might just start running additional experiments and forget about what you learned or a new person comes into the fold, a new colleague and you want to bring them up to speed when they ask like, oh, have we tested this?
A key who bet we have and here's a long document that you can be through with all the information around it from the hypothesis results. So, yeah, I'm a big fan of that approach too and it sounds like we're in agreement there, actually. So, well, thank you. This has been an awesome conversation.
Thank you for helping us kick off this new series. I certainly have, I think you've set a high bar for everyone else who's going to join us here on future lab experiment episodes, but this was really fun, Ashley. So, thank you so much and for all the listeners out there. Thanks for listening.
So, until next time. Thank you.