Four kinds of statement
Most CRO debates go wrong because people mix these up:
| Kind | Example | Ask |
|---|---|---|
| Observation | “Mobile visitors leave most at the shipping step.” | Where’s the data? |
| Interpretation | “They leave because delivery costs surprise them.” | What else could explain it? |
| Hypothesis | “Showing costs on the product page will increase orders.” | How would we test it? |
| Conclusion | “Showing costs earlier increased orders.” | What’s the evidence it was the cause? |
Only a test, or very strong evidence, turns a hypothesis into a conclusion.
How strong is this evidence?
A simple ladder, from weakest to strongest for “this change causes that result”:
- Opinion and best practice. “Everyone does it.” Useful for ideas, useless as proof.
- Qualitative research. Recordings, interviews, surveys. Great for why, weak on how many.
- Analytics. Shows where and how many, but correlation isn’t cause.
- Several sources agreeing. Analytics, recordings and a survey all point the same way.
- A controlled experiment. The best way to test cause and effect on your own audience (Kohavi, Tang and Xu, 2020).
The higher the ladder, the more confident you can be. Most good hypotheses start at step 4.
Turning opinions into tests
When someone says “the homepage is too busy”, ask three questions:
- What did you see that makes you think so? (finds the observation)
- What do you think it causes? (finds the interpretation)
- What would change if we fixed it? (finds the metric)
Now you have the start of a hypothesis, and nobody had to lose an argument.
Recap
Label statements before debating them. Rate evidence by how well it shows cause and effect. Treat opinions as hypothesis starters, not decisions. In WXM, this is the discipline of the Wonder phase: ask better questions before changing anything.
Next lesson: ethics and accessibility.
Practical exercise
Take the last three statements about your website you heard in a meeting. Label each one: observation, interpretation, hypothesis or conclusion. For any 'conclusion', ask: what evidence would we need to be sure?