Use this template for landing-page, offer, campaign, lifecycle and funnel experiments. Complete the plan before launch and the conclusion after the agreed evaluation window.
Link to detailed analysis when needed, but keep the durable decision summary inside the record.
This experiment template keeps the pre-launch hypothesis and judgement rule separate from the post-test result, interpretation and next decision.
Experiment plan
Write the decision rule before launch. Define what would lead to keep, revert, iterate or an inconclusive result.
- Business or customer problem:
- Hypothesis: If we [change], then [audience behaviour] will [move], because [reason].
- Control and treatment:
- Audience and exclusions:
- Primary measure and expected direction:
- Guardrail measures:
- Minimum run or review condition:
- Known risks and concurrent changes:
- Owner and launch date:
Run notes
Record only events that affect interpretation: implementation deviations, outages, tracking changes, audience shifts, campaign changes or unusual external events.
Result and analysis
Avoid choosing a convenient segment after the fact and retelling it as the original test. Label exploratory findings and validate them separately.
- Actual sample or exposure:
- Primary result and uncertainty:
- Guardrail results:
- Segment differences worth investigating:
- Evidence quality and limitations:
- Link to source analysis:
Conclusion and decision
The final sentence should be understandable to a teammate who never saw the dashboard.
- Outcome: positive / negative / neutral / inconclusive
- Reusable conclusion with audience and boundary:
- Decision: keep / revert / iterate / stop / retest
- Next action, owner and date:
- Related prior or follow-up experiments:
Clear answers
Frequently asked questions
Can this template be used for a before-and-after test?
Yes. Replace control and treatment allocation with the exact comparison windows and document seasonality, traffic mix and concurrent changes.
Do we need statistical significance for every experiment?
The analysis standard should match the method and decision risk. If a controlled test is intended to make a causal claim, document uncertainty properly. Directional pilots should be labelled as such.
What happens when guardrail and primary metrics disagree?
Record both and make the trade-off explicit. A primary lift may not justify poorer lead quality, margin, retention or customer experience.
Make this part of the next decision.
Diffined keeps the change, reason, evidence and conclusion together in one private history.
Try the working prototype ↗