January 7, 2026

Happy Tuesday, today’s smart brevity count is: 453 words, 4.18 mins. Edited by Nils Koppelmann.

This week's focus: Set the record straight with strategic experimentation

In today's newsletter, we will go over:

1. Strategic: Kicking off the new year in experimentation
2. Can you guess the results of these 9 A/B-Tests?
3. Your new job in experimentation?
4. Events in 2026
5. Link Lowdown

Excited? Read on.

1 big thing: Kicking off your (new) experimentation program

"New year, new you" – with this mindset I want to share a few resources that I believe can help you 'set the record straight' for your experimentation program and actually achieve the goals you have. I'll also be hosting a free online Experimentation Strategy Workshop (read more below).

1. Run experiments like a system, not like a casino – if you're just building out your experimentation program and starting to run A/B-Tests, then I highly recommend you start by watching this video «How to Build an A/B Testing Program from Scratch»​.
2. Build the strategic foundation for your experimentation program – if you're already running experiments and want to work towards a more strategic approach, check out this video «​A/B Testing was hard until I understood those concepts​» or this one.
3. If you're really just starting out watch my Complete E-Commerce CRO Course 2026.

Post was inspired by Ben Labay & Jonny Longdon's posts e.g. this and this.

2. Can you guess the results of these 9 A/B-Tests

We’re decent at guessing the direction of A/B tests (research with Ron Kohavi and Deborah O’Malley suggests we’re a bit better than random). Jakub Linowski is taking it one step further: can we get better at estimating the size of the effect?

It’s a short, anonymized survey with 9 high-powered experiments. Your guesses will be matched against real outcomes, and Jakub will publish aggregated results once he hits enough responses.

​Take part here and see if how many you get right. (takes ~7 minutes)

I took part myself and curious to see what the actual results are.

3. Your new job in experimentation?

This week’s featured roles:

• ​Senior AI Transformation Manager at Miro (Amsterdam, Netherlands)
• ​Marketing Attribution Experimentation Analyst at Vanguard (USA)
• ​Senior Media Experimentation Manager – Digital Media at Zalando (Berlin, Germany)
• ​eCommerce Conversion Rate Optimisation at VANS (Stabio, Switzerland)
• ​Senior Product Manager II at Strava (Berlin, Germany)

Couldn't find a job that's interesting to you? Check out more here.

3. Events in 2026

2026 has a promising array of events waiting for you – here is a brief overview of some you don't want to miss.

• February 24th 2026, Intent in a (bigger) tent, London UK
• March 17th 2026, Conversionboost, Copenhagen DK
• May 11th & 12th 2026, Speero Circus, London UK
• June 16th 2026, Growth Minded Superheroes, Frankfurt DE

Find more events over at lu.ma/experimentation-events.

4. Link Lowdown

A collection of links to stuff I think worth sharing.

• ​Top Experimentation & A/B Testing Tools for 2026​
• ​Beyond Winning: Spotify’s Experiments with Learning Framework​

That's a wrap – until next week.

– Nils

P.S.:

If you enjoyed this, please consider sharing with a friend. If a friend sent you this, get the next newsletter by signing up here.

Also, for more resources and content follow me on LinkedIn.