Subscription news, activation

Product and context

A subscription news publisher, low hundreds of thousands of subscribers, with a first-quarter cancellation rate the leadership called “the leak”. Onboarding was the agreed villain — loud friction, complained about in cancellation surveys, known to everyone, already scheduled for a rebuild.

Baseline

The baseline confirmed onboarding as loud — 2.2 behaviour, 2.4 perception — and, because it was already scheduled, spent the research budget elsewhere, per the loud-friction playbook. That is where the actual finding was: activation read 2.1 behaviour against 3.8 perception. New subscribers believed they were “using” the product, but the cohort data showed 60% never configured a single topic or newsletter, landed on a generic homepage each visit, and stalled out of the habit loop within three weeks — silently.

Baseline confidence was 0.68, dragged down by guessed inputs on depth of use, which the publisher had never instrumented.

Signals targeted, and why

• Activation — the silent stall carried more modelled churn cost than the loud onboarding complaints.
• Depth of use — guessed at baseline; instrumenting it was itself a treatment, and raised confidence 0.68 → 0.83.

What changed

The scheduled onboarding fix shipped as planned — fewer steps, payment friction removed. Separately, first-session value was pulled forward: the first visit now builds a reader profile from three taps, and the homepage leads with it from visit two. Depth-of-use instrumentation shipped in week one, before any treatment, so the effect would be measurable.

Rescore

Ten weeks later: activation-event completion rose from 40% to 61% of new cohorts, week-three retention improved four points, and the composite moved 2.6 → 3.2 at 0.83. Onboarding, post-rebuild, moved to watch — behaviour improved, perception still lagging, as expected for a surface users had complained about for a year.

What did not work

A re-engagement email sequence aimed at stalled subscribers — the marketing-side fix for the same leak — was tested against the product-side fix and did nothing durable: opens were fine, return visits spiked for a day, week-three retention was unchanged. The stall was a product state, not an attention problem, and no email fixes a homepage that shows every reader the same thing. The sequence was retired.

Elapsed time

Ten weeks from baseline to rescore, running alongside the already-planned onboarding rebuild. The activation fix itself was one squad for one sprint.