November 18, 2025

Happy Tuesday, today’s smart brevity count is: 428 words, 3.25 mins. Edited by Nils Koppelmann.

This week's focus: Why Most Experiments Don't Move the Needle

In today's newsletter, we will go over:

1. Strategic: The Misalignment Tax
2. Your new job in experimentation?
3. Upcoming events in 2025

Excited? Read on.

In the last edition, I shared how to build experimentation systems. Today, I want to tackle a different problem: 85% of experiments aren't connected to business goals.

You can have a perfect system running flawed experiments. Let's fix that.

1 big thing: The Misalignment Tax

Strategic Insights for Experimentation Leaders

Here's a stat that should concern every experimentation leader: 85% of experiments aren't connected to business goals¹.

Teams are running tests. They're following best practices. They're pre-registering hypotheses and avoiding p-hacking. But they're optimizing the wrong things.

The result? Experimentation programs that feel busy but deliver disappointing business impact.

Why this happens:

Most teams start with solutions instead of problems and while these aren't inherently bad tests they are disconnected from what actually matters.

This transforms experimentation from random wins into the vehicle that drives learning and growth.

When your tests are strategically aligned, each experiment – winner or loser, really doesn't matter – generates insights that inform the next test and more importantly the direction you're going. Random testing doesn't compound. It just accumulates.

So the strategic question is: Are you just measuring experiment velocity or learning velocity?

¹ Data collected by Bjarn Brunenberg from participants in a webinar on the topic

2. Your new job in experimentation?

This week’s featured roles:

• ​Staff Product Analyst (Experimentation) at Welltech (Spain, Cyprus, Poland, Ukraine)
• ​Product Analyst at Aroundhome (Berlin, Germany)
• ​Senior Staff Machine Learning Engineer, Relevance and Personalization at Airbnb (USA)
• ​Product Experimentation Lead at Betsson Group (Malta)
• ​Senior Customer Insights Analyst at SIXT (Lisbon, Portugal)

Couldn't find a job that's interesting to you? Check out more here.

3. World class events

Upcoming conferences in in 2025:

• 21st-23rd November 2025, Conversion Hotel in Texel, Netherlands
• 9th December 2025, Experimentation Elite in London, UK
• 11th December 2025, Berlin Experimentation Meetup in Berlin, DE

Find more events over at lu.ma/experimentation-events.

That's a wrap – until next week.

– Nils

P.S.:

If you enjoyed this, please consider sharing with a friend. If a friend sent you this, get the next newsletter by signing up here.

Also, for more resources and content follow me on LinkedIn.