Optimizers run tests on websites to find winning combinations and get better results. So it’s natural that people who are new to Conversion Rate Optimization can end up focusing on the site rather than the people who are using it.
But the site it not what you’re testing.
What you’re testing is the audience.
The site is something we can control as optimizers. We can’t control audiences. So we change what we can control (the site) in the quest for better results.
Ultimately, though, the audience holds the power. A winning version convinces the audience to take the action the optimizer desires. If the audience is unconvinced, then the test will fail.
Implications
Audiences are not static. They change between sites and over time.
Testing Results Are Not (Necessarily) Transferable
A change that works on one site won’t always work on other sites.
The audience is different for every site and the environment (eg what the site sells, competitive landscape, site structure) is different. These differences introduce variables that change how an audience reacts to a test.
So what worked on one site might not work on another.
There are some best practice exceptions to this, eg placing the call to action prominently usually helps. But even then there’s no guarantee. Most optimizers have stories of best practice failures.
Because the audience and environment changes from site to site, it is not possible to guarantee the outcome of a site change, even if it has worked elsewhere.
Instability
Your audience is going to change over time.
For instance, audience demographics, cultural expectations, motivations, and expected site behavior can all shift.
So what worked once might not work again. And likewise, something that failed in the past might work in the future.
Because you are testing the audience and the audience changes over time, testing results can be unstable over long periods of time.
Managing a Changing Environment
If the element you’re testing – the audience – changes constantly, how do you manage that? The short answer is by being knowledgable and flexible.
Know Your Audience Now
To develop successful site changes, you need to know as much as possible about your audience.
This is not an exhaustive list, but a few places to start:
- Gather quantitative and qualitative data and think about what it tells you about your audience and their motivations.
- Have a critical look over the site as it currently stands. Where could your audience be getting lost or confused? Does the data back you up?
- Look at sites that are solving similar problems. While you should study your competitors, also look outside of your competitive space to find new solutions.
Gather as much information as you can to understand your audience and make predictions on how they are likely to react to site changes.
By knowing your audience well, it is easier to design successful tests.
Include “Why” in Your Hypotheses
When you develop hypotheses for site changes, always know why a change should work based on what you know about your audience.
If the test succeeds, then it helps validate your ideas about the audience.
If you test fails, then it’s worth taking another look at your assumptions. Have you misunderstood your audience? What new facts have you learned about the audience?
By including why a change will work as part of your hypothesis development, you stay focused on testing your audience rather than your site.
Evolve With Your Audience
Your audience will change over time.
Most sites see a slow progression of changes, rather than big, sudden shifts.
If what you need to manage is gradual change, approaching testing with humility can be a very successful strategy. Assume that you always have something new to learn.
Stay focused on what the data tells you. With each new data point, your understanding of your audience should change and deepen. Use that information to run successful tests even as your audience changes.
The goal is not to stake out a single correct position, but to recognize the audience as an ever-changing testing environment and to successfully adapt.
Conclusion
Conversion Rate Optimization isn’t about testing sites, it’s about testing audiences. Successful site testing is dependent upon knowing your audience well.
When constructing tests and hypotheses, rely on what you know about your audience. And always be eager to learn more.