April 21, 2026

Happy Tuesday, today's smart brevity count is: ~850 words, ~3.5 mins. Edited by Nils Koppelmann.

This week's focus: Having the right team structure isn't enough — here's what kills programs after the foundation is in place.

In today's newsletter, we will go over:

1. Strategic: Why Experimentation Programs Stall
2. Tactical: The Program Health Check
3. Your new job in experimentation?
4. Events

Excited? Read on.

1 big thing: Why Experimentation Programs Stall

Strategic Insights for Experimentation Leaders

Over the past two weeks we've covered how to structure your experimentation function and which roles you actually need. But here's the uncomfortable follow-up: most programs that have both of those things still stall.

Not at the start — at the start there's energy and momentum. The stall comes later. Six months in, twelve months in. Tests are still running, but something feels off. Results aren't moving things forward. Leadership is less engaged. The team is busy but the program isn't progressing.

The cause is almost never what people think it is. Teams go looking for better tooling, more headcount, a new framework. Usually it's none of those. It's one of four failure patterns — and they're remarkably consistent across organisations.

Pattern 1: The Decision Gap

Tests finish. Nobody ships. Results sit in a dashboard or a Slack thread, and two weeks later the team is already onto the next experiment without having made a clear call on the last one.

This isn't a data problem. It's an ownership problem. If there's no clear decision owner for each experiment — someone whose job it is to say "ship it" or "kill it" — results accumulate into a backlog of unresolved questions. The program keeps moving but nothing actually changes.

Pattern 2: The Velocity Trap

The backlog is long, but the number of live experiments isn't growing. Ideas pile up — from stakeholders, from the team, from research — but tests take forever to launch. Engineering dependencies, design bottlenecks, QA loops. The experimentation function becomes a holding pen for good intentions.

Low velocity compounds fast. If you're running two tests a month where you should be running eight, your learning rate is four times slower. By the time you notice, you're a year behind where you could have been.

Pattern 3: The Trust Deficit

This one is slow and quiet. Leadership stops referencing experiment results in decisions. Stakeholders start running "quick tests" outside the formal process. The experimentation team keeps doing rigorous work that nobody reads.

Trust erodes when results feel unreliable — often because of real methodology problems, but sometimes just because results were communicated poorly. Either way, once it's gone it's hard to rebuild without a visible win that leadership can point to.

Pattern 4: The Single Point of Failure

One person holds the program together. They know where everything is, maintain the standards, manage the stakeholder relationships, and keep the momentum going. Then they leave — or go on parental leave, or shift to a different project — and the program quietly collapses.

This is the most preventable of the four patterns and the most common. Programs built around people rather than process don't survive transitions.

Most stalls are caused by one of these four things. Occasionally two at once. Rarely all four simultaneously — though if you're at all four, the program probably isn't being called a program anymore.

The reason it's useful to name them precisely: the interventions are completely different. A trust deficit requires a different fix than a velocity trap. Misdiagnosing the pattern leads to solutions that look right but don't work.

2. The Program Health Check

A five-question diagnostic to identify which stall pattern you're in

Run through these five questions about your current program. Each one maps to one of the four stall patterns. Be honest — the point is to find the problem, not to score well.

1. What's your average time from test completion to a documented ship/kill decision?
→ If it's longer than two weeks, or you don't track this: Decision Gap.

2. How many experiments are currently waiting to launch vs. actively running?
→ If waiting > running, or you can't answer: Velocity Trap.

3. When did a senior stakeholder last reference an experiment result in a business decision — unprompted?
→ If you can't remember a recent example: Trust Deficit.

4. If your experimentation lead left tomorrow, what would break — and for how long?
→ If the answer is "a lot, for a long time": Single Point of Failure.

5. What's your win rate over the last quarter, and does it match your team's intuition?
→ A win rate below 20% suggests methodology problems. A win rate above 70% suggests tests that aren't challenging enough. Either can mask deeper issues.

One or two questions triggering concern is normal. Three or more is a signal that the program needs structural attention, not just more experiments.

3. Your new job in experimentation?

This week's featured roles:

• Marketing Data Scientist – Measurement at Just Eat Takeaway.com (Amsterdam, Netherlands)
• Senior Customer Data Science – Solutions / Experimentation at Datadog (London or Paris)
• Online Experimentation And Data Lead at The Home Depot (Canada)
• Conversion Rate Optimization Manager at Jooble (Ukraine, Remote)
• Technical Product Manager at Booking.com (Manchester, UK)

Couldn't find a job that's interesting to you? Check out more here.

4. Events

• Apr 22, Experimentation Unite Vienna (by Kameleoon), Vienna AT
• Apr 22, The Intersection of Healthcare and eCommerce (by Convert.com), Online
• Apr 27–30, Marketing Analytics Summit, Santa Barbara CA
• Apr 28, How Are Experimentation Agencies Adapting to AI in 2026? (AMA by Convert.com), Online
• Apr 28, Berlin Experimentation Meetup #13, Berlin DE

Find more events over at lu.ma/experimentation-events.

That's a wrap – until next week.

– Nils

P.S.:

If you enjoyed this, please consider sharing with a friend. If a friend sent you this, get the next newsletter by signing up here.

Also, for more resources and content follow me on LinkedIn.