Hypotheses are a necessary tool for site testing. A hypothesis can be thought of as a short story that predicts how users will react when a site change is introduced.
Hypotheses are valuable because they keep you focused. If you can’t say what impact you expect a change will have or why you believe the change will work, it’s a good time to dig into the evidence and make sure you really understand what’s going on. Writing a hypothesis is an early test of your idea to ensure it holds together before pushing changes live on the site. Hypotheses also make it easier to prioritize tests and share ideas with other people in the company.
The elements of a hypothesis include: the area or functionality; what you want to change; why it matters to your audience; the metric you’re tracking; and the predicted outcome.
Let’s run through a quick example of how this all comes together.
Suppose you run a social media platform. You want your audience to interact with the platform more frequently both as creators and viewers of content. You believe the best way to accomplish this is to increase the number of photos people upload.
Area or functionality: Your audience’s interaction with the site’s interface, specifically uploading images.
What you want to change: The strength of the button for image uploads so people are more likely to see it.
Why it matters to the audience: Focus groups have indicated that customers love sharing photos over the social media network as it is quicker and easier than writing an update but it still feels like keeping in touch. However usability tests have shown that customers have trouble finding the button with which to upload photos. (Why you think it matters won’t be in the final hypothesis statement but it’s a very important step when building hypotheses. If you can’t convincingly explain why you think an audience will react well to the change you are proposing, your hypothesis is more likely to fail.)
Metric: The number of photo uploads divided by the number of total updates on the social media platform. Right now, photo uploads account for 3% of all updates.
Predicted outcome: A 5% increase in photo uploads.
Pulling all these pieces together, the hypothesis is: “Making the photo upload button stronger will increase photo uploads by 5%.”
If you have a great idea but are having trouble creating the elements of a hypothesis, it may be because you need to narrow down what you’re testing. It is hard to create an effective hypothesis, and effective site tests, around an idea that is too broad. Another possibility is that your idea is sound but you need to refine it a bit more before you can act on it. By digging into available evidence, you can develop more insight into what will work and why.