The short answer
Your boss, client or founder has an idea they’re sure about. You think the evidence points elsewhere. Arguing usually loses, and even when you win, you’ve made testing feel like a fight.
So let the boss’s idea run first. Turn it into a proper hypothesis, test it fairly, and report it honestly. Whatever the result, you’ve shown that the test decides. That buys you the right to test your own evidence-led ideas next.
This is a practitioner opinion, and the one our expert reviewer holds most strongly: politics slows programs down more than statistics does.
Why it works
- Trust beats being right. People support a process they’ve seen treat their idea fairly.
- Velocity stays up. A test running is worth more than a week of debate. Debate produces zero learnings.
- You might be wrong. Senior people often know things about customers that never made it into the data. Sometimes their idea wins.
- A losing test persuades better than you can. Experiments are valuable exactly because intuition, including expert intuition, is often wrong (Kohavi, Tang and Xu, 2020).
How to do it well
1. Make it a real hypothesis
Ask, with genuine curiosity: “What do you expect to happen, and why?” Write their answer into the hypothesis format:
Because customers tell sales that our pricing is confusing, we believe a simpler three-plan pricing page will increase demo requests, because it makes choosing easier.
Now it’s testable, and the “because” tells you what you’ll learn either way.
2. Agree the decision rule before launch
“If demo requests go up and lead quality holds, we ship. If not, we keep the current page and look at the sales feedback again.” Agreeing this upfront is what makes the result accepted later.
3. Build it quickly
Don’t let the boss’s idea wait in a three-month queue. A clean, simple version that’s live this week is better than a perfect one in spring. See done is better than perfect.
4. Report it with respect
Win or lose, credit the idea and focus on the learning. “Your instinct about confusion was right; the three-plan layout wasn’t the fix” keeps the person on side.
When not to let it run
- It’s deceptive or harmful. Fake urgency, hidden costs, dark patterns. Say no and explain why. See ethical CRO.
- It would block the testing slot for months. Then scope it down or run a cheaper version.
- The page doesn’t have the traffic. A test that can never reach its sample size wastes everyone’s time. Explain that with the sample size calculator.
Common mistakes
- Testing it half-heartedly so it loses. People notice, and trust is gone.
- Skipping the hypothesis because “it’s just the boss’s idea”. Without the “because”, you learn nothing.
- Gloating when it loses. The goal is a program everyone believes in, not a scoreboard.
What to do next
Once ideas are flowing, rank them fairly with CRO prioritization, and report results in a way that lands: test reports people actually read.