April 14, 2026
Happy Tuesday, today's smart brevity count is: ~1600 words, ~6 mins. Edited by Nils Koppelmann.
This week's focus: The most common source of dysfunction in experimentation teams isn't bad methodology — it's unclear ownership.
In today's newsletter, we will go over:
- Strategic: The roles your experimentation team actually needs
- Your new job in experimentation?
- Events
Excited? Read on.
1 big thing: The Roles Your Experimentation Team Actually Needs
Strategic Insights for Experimentation Leaders
Last week we talked about org models — centralized, decentralized, federated. The obvious follow-up question: okay, so who do I actually need on the team?
The most common source of dysfunction in experimentation teams isn't bad methodology. It's unclear ownership.
When everyone does "some experimentation," nobody fully owns any of it. Tests launch without proper design. Results get misread. Decisions stall while people wait for someone else to take the lead.
A mature experimentation function has five distinct roles. In large orgs, these are separate people. In smaller teams, one person might cover two or three. But the responsibilities still need to be assigned — even if they land on the same desk.
Experimentation Lead — owns the practice itself. Methodology standards, program roadmap, methodology disputes, senior stakeholder relationships. They don't run every test — they ensure every test meets the bar.
Analyst / Data Scientist — the technical backbone. Statistical design, sample size calculations, instrumentation QA, results interpretation. Crucially: they should be involved before an experiment launches, not just after it finishes.
Product or Growth PM — owns the backlog and prioritization. Connects experiment output to product strategy. The best PMs treat the experiment backlog exactly like a product backlog: ruthlessly prioritized, with clear hypotheses attached to each idea.
Engineer — owns implementation. Variant builds, tracking instrumentation, setup QA. Engineers who understand experimentation are a significant multiplier — they catch instrumentation errors before a test runs, not after.
CRO Specialist (where applicable) — primarily a test ideation role, drawing on behavioral research and customer data. Not every team needs this, but for high-volume consumer teams it compounds quickly.
The "everyone does some experimentation" failure pattern
When roles aren't clear, you get a specific breakdown: the analyst is doing design work, the PM is running their own analyses, engineering is consulted after the fact, and nobody has a clear picture of what's in flight.
The fix isn't to centralize everything. It's to distinguish between who owns a decision and who contributes to it.
Here's a RACI-lite for each experiment phase:
| Phase |
Owner |
Contributors |
| Hypothesis & scoping |
PM |
Analyst, CRO |
| Statistical design |
Analyst |
Exp. Lead |
| Implementation & QA |
Engineer |
Analyst |
| Results analysis |
Analyst |
PM, Exp. Lead |
| Decision & shipment |
PM |
Exp. Lead |
| Socialization & learning |
Exp. Lead |
PM, Analyst |
Owner = one person makes the call. Contributor = input required, not the decision.
Run this against your current team. If more than one person has "owner" on any phase, you have a clarity problem. If a phase has no owner, you have an accountability gap.
When you can't hire
Most teams can't staff all five roles from day one.
One person: analyst-lead hybrid. They own methodology and program oversight. PM responsibility falls to whoever owns the product roadmap — make it explicit even if informal.
Two people: split analyst and program lead. The lead handles stakeholders, prioritization, and socialization. The analyst handles design and analysis.
Three or four: start to federate. One analyst per major product area, reporting to a central lead. This is the early federated model — and it scales.
The mistake is hiring generalists who "do experimentation" without defining which responsibilities they own. Two people with unclear roles will be less effective than one person with total clarity.
The implication: before you hire another experimenter, map your current team against these five roles. Identify which phases have no clear owner — and assign them explicitly, even informally. That conversation is worth more than a new headcount.
2. Your new job in experimentation?
This week's featured roles:
Couldn't find a job that's interesting to you? Check out more here.
3. Events
Find more events over at lu.ma/experimentation-events.
That's a wrap – until next week.
– Nils
P.S.:
If you enjoyed this, please consider sharing with a friend. If a friend sent you this, get the next newsletter by signing up here.
Also, for more resources and content follow me on LinkedIn.