Company deep dives

Spotify interviews

Sweden

Experiments

“Most of our learning doesn’t come from wins — it comes from discovering what not to ship.”

Michael Bellato, Mårten Schultzberg and Sebastian AnkargrenSpotify data scientists

Spotify Engineering · 2025-09-23

Learning comes from what not to ship

Decide whether a 10% onboarding lift is ready to ship.

Example

“You can have a 0% win rate but still get huge business value from experimentation if you’re learning and mitigating the risks of making bad decisions.”

Michael Bellato, Mårten Schultzberg and Sebastian AnkargrenSpotify data scientists

Spotify Engineering · 2025-09-23

A 0% win rate can still pay

Decide whether one winner among 20 AI-written headlines should roll out.

Example

“By articulating the heuristics that govern your decision-making process, you can design and analyze the experiment in a way that respects how you make decisions in the end.”

Mårten Schultzberg, Sebastian Ankargren and Mattias FrånbergSpotify data scientists

Spotify Engineering · 2024-03-05

Design around how you decide

Decide whether a 10% onboarding lift is ready to ship.

Example

“It’s OK if a hypothesis is incorrect; it’s not OK if we spend all that time and effort and don’t have clear results.”

Nik GoyleProduct Manager, Spotify

Spotify Engineering · 2023-06-15

Wrong is fine; unclear is not

Decide whether a 10% onboarding lift is ready to ship.

Example

What AI changes in the work

“For example, new AI features may bring added costs, so we need reliable estimates to weigh benefits against trade-offs.”

Michael Bellato, Mårten Schultzberg and Sebastian AnkargrenSpotify data scientists

Spotify Engineering · 2025-09-23

Weigh AI features’ costs

Decide whether AI share cards that doubled sharing should roll out.

Example

Last reviewed