CRO Hypothesis Template: How to Write a Hypothesis Worth Testing

"Let's test a new headline" is an idea. "Because most visitors who reach the pricing table leave without clicking, we'll add the monthly price to the hero for paid search visitors and expect more clicks on Start trial" is a hypothesis. The second one tells you what to build, who sees it, what to measure, and what you've learned whether it wins or loses.

This guide gives you a template to copy, explains each part, shows weak and strong examples for landing pages, and covers how to use the same template when you can't run an A/B test.

Why bother writing it down

Ideas are cheap and hard to judge. Ronny Kohavi, who led Microsoft's experimentation platform, summed up years of controlled experiments with "we are poor at assessing the value of our ideas" and "the less data, the stronger the opinions" (Kohavi, Online Controlled Experiments: Introduction, Pitfalls, and Scaling, 2016). A written hypothesis is a small defense against both. It forces you to say where the idea came from and how you'll know it was wrong.

It also makes a losing change useful. "The new hero lost" teaches you little. "Showing the price early didn't move clicks for paid search visitors" is something you can use on the next idea.

The template (copy this)

Because we saw [evidence: source and what it showed], we will [one specific change] for [audience: device, traffic source, new or returning], and expect [primary metric] to [go up or down], without [guardrail metric] getting worse. We'll decide after [sample size or time window], and if it loses we'll learn [what a loss would tell us].

The parts, one at a time

1. Because: the evidence

This is the reason the hypothesis exists. Good sources are things you can point to: a scroll map showing most phone visitors leave before the button, a click map with heavy taps on a plan name that isn't a link, a recording pattern, a support ticket theme, a form field where people stop. If the only evidence is "I think", you have a guess, not a hypothesis. That's fine for brainstorming, but it should rank low when you prioritize your CRO ideas.

2. We will: one specific change

Describe it clearly enough that a developer could build it without a meeting. "Improve the form" isn't a change. "Remove the Company and Phone fields from the demo form" is. Microsoft's experimentation team recommends keeping the hypothesis simple: if a change is complex, break it into a series of simple changes, each with its own hypothesis, so you can measure each part (Microsoft Research, Patterns of Trustworthy Experimentation: Pre-Experiment Stage). Kohavi's talk gives a reason: if each new feature has a 10% chance of a serious problem, an experiment that combines seven of them fails about half the time (1 - 0.9^7 = 52%) (Kohavi, 2016).

3. For: the audience

Name who sees the change: all visitors, phones only, paid search traffic, first-time visitors. The evidence usually tells you. A problem seen on the mobile scroll map is a mobile hypothesis. Our guide to mobile vs desktop heatmaps covers how to spot device-specific issues.

4. Expect: one primary metric and a direction

Pick the single number that decides the result before you start, and say which way it should move. Choose one close to the change: clicks on the button you moved, completions of the form you shortened. Microsoft's team adds that the hypothesis should be falsifiable by the metrics you choose. A claim that a change will "improve branding" is hard to prove or disprove without a metric that measures brand (Microsoft Research).

5. Without: a guardrail

A change can win on its metric and still hurt the business. A shorter form can bring more leads and fewer qualified ones. A louder discount banner can lift add to cart and lower order value. Name one or two numbers that must not get worse. Microsoft recommends a core set of guardrail metrics on every experiment for this reason (Microsoft Research).

6. Decide after: the rule, written in advance

Say how long the change runs or how many visitors you need before you look, and what result counts as a win. For an A/B test, that comes from a sample size calculation. For a before-and-after change, it's equal time windows on the same weekdays. Writing it down first is what stops you from stopping early on a good day.

Weak vs strong: three examples

The numbers in these examples are illustrative. Use your own.

What the strong versions have in common: a real source, one change, a named audience, one metric with a direction, and something to protect.

Common mistakes

• Writing the hypothesis after the result. If it's written after, it's a story, not a test.
• Bundling changes. New headline, new image and a new button in one go tells you nothing about which one mattered.
• Choosing a metric far from the change. Revenue is what you care about, but a button change on one page rarely moves site-wide revenue enough to see. Pick the closest metric, guard the important one.
• No way to lose. If no result would make you say "we were wrong", the hypothesis isn't testable.
• Putting bugs through the process. A broken form isn't a hypothesis. Fix it first.

Without an A/B test

Many sites don't have the traffic for a clean A/B test on every idea. The template still works. Ship the change, compare equal windows before and after, and read the result against the rule you wrote down. Note anything else that changed in the same period, like a campaign or a price change. Our guides to CRO without A/B testing and to proving CRO results to clients cover how to do the before-and-after comparison honestly.

Where Cortality fits

The hardest part of a hypothesis is the "because". Cortality gives you that part: one tag measures how real visitors use every part of your page, including what they see, skip, tap and stop at, on each device. You get a ranked list of fixes, each with the evidence and the exact change to make, and after you ship a fix Cortality re-measures it and tells you whether it worked. It works alongside Google Analytics and Microsoft Clarity rather than replacing them. See how it works or what a finding looks like.

For a quick first look, our free page leak scan takes about a minute and needs no install. It's a prediction from our attention model, not data from your real visitors, so use it to decide where to look first.

Plans are $99, $249 or $499 a month (Starter, Growth and Scale). Start your free week, and you see your first report before you pay. You save a card at signup, and nothing is charged until 3 days after your first report. Cancel before then and you pay nothing, and any charge can be refunded within 14 days. How the free week works.