Survivorship Bias.

01Overview

Survivorship bias is what happens when an analysis quietly drops everything that didn't make it. You read about the startup that pivoted at the perfect moment, not the ten that pivoted into a wall. You watch the redesign that lifted retention, not the eleven that didn't ship the case study.

For designers, it's the bias that makes "best-in-class" research feel rigorous when it's actually a survey of survivors.

02Detailed explanation

The textbook story belongs to the statistician Abraham Wald, in 1943. The U.S. Air Force showed him a diagram of where returning bombers had been hit and asked where to add armour. The intuitive answer: reinforce the most-hit spots. Wald's answer: reinforce the spots with no bullet holes. Planes hit there hadn't come back to be diagrammed.

The pattern repeats endlessly in product work:

• Customer interviews with current users tell you why current users stayed. They tell you almost nothing about why most prospects bounced.
• "What makes successful onboarding?" studies look at the cohort that completed it. The cohort that didn't is the cohort with the lesson.
• Best-of decks at conferences are a museum of survivors. The graveyard is bigger, better-organised, and rarely on stage.

03Why it exists

Survivors are visible. Non-survivors aren't around to be sampled — they churned, refunded, never signed up, never wrote a postmortem. Meanwhile, the brain's pattern-matcher loves a clean story: "they did X and won." Adding the missing comparison — "…but most of the people who did X also lost" — takes deliberate effort.

04Effects on users

Users carry their own version. They model "people like me" from the people they can see — which skews toward loud success stories and visible failures, missing the quiet middle.

• Product reviews look strong because the people who gave up never wrote one.
• Testimonials emphasise transformation stories; the median user got mild, useful value and forgot to mention it.
• "You can do this too" pitches feel achievable because every counterexample stayed home.

05Effects on designers & teams

Three places where this bias quietly costs you:

• Best-practice envy. Teams imitate the patterns of household-name products without checking whether those patterns are causally responsible — or just things successful companies happen to do.
• Power-user research. Sessions with five enthusiastic users feel revealing and aren't. The product is what it is partly because they're the ones still around.
• Case-study cargo cult. "We A/B tested a green button and conversion went up 27%." Cool. The other team that tested a green button and saw no lift didn't write a case study.

06Practical takeaways

• Recruit churned users. They cost more to find and they're the most valuable interview you'll do this quarter. Budget for it.
• Ask "who's missing?" before "what's the pattern?" In any synthesis, name the population your data can't describe.
• Track non-conversions. Funnel analytics show the people who started. The interesting cohort is the one that never started — and they don't show up in your funnel by definition.
• Read failure post-mortems. They're rarer, harder to find, and pound-for-pound more instructive than success stories.
• Distrust the case study with the round number. A 27% lift, neatly attributable to one change, is usually a survivor.

07Design examples

08Ethical risks

Survivorship bias quietly punishes users whose stories are inconvenient. The single mum who couldn't finish onboarding, the visually impaired user who bounced at step two, the small-business owner who churned in week three — these are exactly the people whose feedback would steer the product toward more people like them. They are also the hardest people to schedule a Zoom with.

Design teams that take this seriously build the recruiting muscle to find non-survivors. The ones that don't end up designing for an increasingly narrow slice of humanity.

09Related biases