A thought experiment is an experiment that you run without making site changes. You attempt to prove, disprove or refine an idea with existing evidence.
When you run thought experiments you can find issues in your hypotheses early before you make site changes. This saves you design and development time, and you also avoid burning testing time on a site change that won’t work.
By running thought experiments, you increase the chances that when you roll out changes, they will be successful.
To walk through an example… Suppose the company you work for sells artwork that ranges from cheap prints to expensive reproductions. Most of the orders are for cheap prints. The assumption within the company is that most revenue is driven by the low cost, high volume products because so many of them sell.
The initial hypothesis is roughly: “Change the site to heavily promote the low cost, high volume products to increase revenue”.
As a thought experiment, you decide to test whether the low cost, high volume products make up the majority of the revenue.
When you look at the order numbers, you find that most purchases are for low cost items. But when you look at the revenue, there are two peaks. One peak corresponds to the low cost sales. The other peak corresponds to the high cost sales. While there are far fewer high cost sales, the margin is so much higher they bring in a lot of revenue.
This suggests that instead of one primary audience behavior there are two – one behavior pattern is to purchase low cost products and the other is to purchase high cost reproductions.
To further refine your thought experiment, you pull out information on who these high cost purchasers are. You discover that they are concentrated in urban areas, specifically Los Angeles, San Francisco and New York. The types of artwork that they most frequently purchase are large, abstract paintings. The paintings are purchased by individuals, not businesses. It appears likely that people are purchasing these paintings to decorate their apartments or homes.
By following a thought experiment you’ve found out a lot about a profitable audience segment that hadn’t been identified. You also have some new ideas for how to increase revenue, e.g. make high cost artwork more visible on the site, expand the types of artwork offered, start a weekly feature about how to decorate your urban apartment with artwork.
Thought experiments can help you find patterns and audience behaviors that have been overlooked. By running thought experiments before making site changes, your site changes are more likely to be successful because they will be informed by evidence rather than assumptions.