Five skills

Each comes with a worked example you can play through before you see a strong answer.

1. Skill 1Read the test before you ship the winnerRestate a lift as counts, compare the sample with the plan, and call an underpowered result inconclusive.Worked example: The new onboarding shows a 10% lift. Would you ship it?
2. Skill 2Treat cheap variants as guesses, not winsCount your comparisons, set a bar that fits them, and rerun the best variant before you trust its lift.Worked example: One of 20 AI-written headlines won. Would you roll it out?
3. Skill 3Split retention by cohort before blaming the productSplit a blended rate by cohort and channel, and judge each channel on its cost per retained user.Worked example: Retention fell after an AI launch. Would you roll it back?
4. Skill 4Choose the metric that predicts the outcomeAn activation metric must predict the outcome you care about. Check that before it becomes a goal.Worked example: Plan generation jumped to 71%. Should it be the goal?
5. Skill 5Measure the whole loop, not the step that movedJudge a loop by the new active users it produces, not by the step that moved.Worked example: AI share cards doubled sharing. Would you roll them out?

Draft · not yet reviewed