Case study

Seekho

Edutainment OTT · $100M+ ARR · 100M downloads

Pavo built a pLTV model for Seekhoto improve M3 retention by +8.23%

In seven days, Pavo turned Seekho's Day-0 ad signal from a watch-time rule into a model-built one: in offline evaluation it flags a baseline-sized slice of trials that retains a relative 8.23% higher to month 3, packaged to activate through Meta's Conversions API and confirm with an online A/B.

Monthly active users: 25M
Downloads: 100M
Annual recurring revenue: $100M+

+8.23%M3 retention lift · offline eval
≈35%Coverage, matched to baseline
7 daysOnboarding to a shippable signal

The problem

The signal Seekho sent Meta was the wrong one

Seekho runs paid acquisition on Meta and Google. Both platforms are look-alike engines: you send them a Day-0 conversion signal that says this new user is worth chasing, and they spend your fixed budget finding more people like that one. So the quality of who you acquire is capped by the quality of that signal.

Seekho's signal was ten or more minutes of watch time on Day 0. That cohort is roughly a third of trials, and its month-3 (M3) retention is the baseline every new signal has to beat. The catch: watching ten minutes on the first day only weakly predicts still being subscribed ninety days later, so the platforms were optimizing for the wrong user.

Improving LTV meant sending a sharper Day-0 signal, under three constraints that make it hard:

01Coverage must stay near 35% of trials. The platform needs enough positive events each day to learn from; too strict a signal starves it and delivery breaks.
02Day-0 features only. The signal fires right after acquisition, so it cannot depend on anything a user does on Day 1 or later.
03Per ad-set, and CAPI-ready. Meta optimizes at the ad-set level, so the signal has to be expressible for each ad-set and shipped through its Conversions API.

Finding a sharper signal is easy if you are allowed to shrink the cohort. Finding one that is sharper, still covers about 35%, and uses only Day-0 data is the actual problem.

The approach

Absorb the data, search widely, distill to one signal

Pavo ran rules first and models next, spending the cheap, legible approaches before the expensive ones, and validating each step before trusting it.

01
Absorbed the system knowledge, validated the baseline
- Connected to Seekho's warehouse and translated tribal knowledge into a Day-0 pre-aggregate table: about 41 columns across trial dates, Day-0 engagement, show-level behavior, and M1/M2/M3/M6 retention flags, for every March–May 2025 trial.
- Reproduced the baseline retention from source before trusting it.
02
Scoped a bounded suite of approaches
- Single signals, composite signals, a predictive model, and further out, survival and journey-cascade models.
- Named the whole search up front so no promising branch was skipped and no expensive one was run early.
03
Searched for better heuristics
- 1-predicate: information-value sweeps, decision-tree (CART) rules, an engagement score. The best single rule moved M3 retention about 1%, not enough to ship.
- 2-predicate: exhaustive pairs scored by information value and CART. Sharper, but each rule covered only 20–27% of trials.
- Unions of the top rules, per ad-set, to claw coverage back toward 35%.
04
Built a Day-0 propensity model
- Trained a logistic regression on Day-0 features only to predict each user's M3-retention probability.
- Picked the probability threshold that flags the top 35% of users, then distilled the model into a single SQL rule so it can ship as a signal.
- Trained ad-set-specific models for the top 7 ad-sets.
05
Validated offline, day by day
- Checked coverage across all 31 days: the model cohort stayed within a few percent of the baseline cohort's size every day, so the daily feedback cohort would not swing and break delivery.
06
Packaged to activate and A/B online
- Packaged the per-ad-set SQL signals to emit through Meta's Conversions API, staged for a controlled online A/B at constant spend and unchanged creatives, the next step after this engagement.

The suite

From single rules to multi-task models

Pavo scoped a bounded suite of approaches and ran it cheapest-first, from single-predicate rules to multi-task models. Every tier was explored; the Day-0 logistic model, distilled to one SQL rule, won on the evidence.

41-col pre-aggMar–May 2025 · all trials

100+approaches tried

IV / KS + CARTheuristic search

Offline replay31-day day-wise ✓

7 ad-set modelstop Seekho ad-sets

Methods explored

Information ValueKolmogorov–SmirnovDecision Tree · CARTEngagement scoreLogistic regressionOrdinalSurvivalJourney cascadeMulti-taskBehavioral personas

011-predicate

Single signals

IV sweep · CART · engagement score

first_video_seconds_watched > 55

avg_minutes_per_video > 1.72

first_video_duration > 176

Explored · best ≈ +1%

022-predicate

Composite rules

watchtime ≥ 7 & completion ≥ 95%+3.5%

watchtime ≥ 7 & avg_min/video ≥ 2+6.3%

80%-completed ≥ 1 & first_sec ≥ 60+8.0%

20–27% coverage · per-ad-set unions restore ~35%

Built

03Propensity model

Day-0 logistic model → SQL

Logistic regression · 20 Day-0 features

completion_pctwatchtimeavg_min/videodistinct_showsvideos_80pctdropped_earlyhad_d0_activity…+13

Threshold @ top-35% → distilled to one SQL rule
31-day offline validation ✓ · 7 ad-set-specific models

Best · +8.23%, coverage matched

04Advanced models

survivalp(retention > t)

journey cascadep(m3|m2)…

multi-taskd1/m1/m2/m3

ordinal{d1,m1,m2,m3}

personasbinger

Explored · none beat the model

+8.23%

M3 retention lift · coverage matched to baseline · offline evaluation

ExploredBest on the evidence

← basicincreasing sophistication →advanced

Every approach Pavo explored, ordered by increasing sophistication. Each tier carries its real work; the lime marks the one that won. Lifts are relative to baseline M3 retention; absolute rates stay Seekho's.

The search

Sharper rules existed, but only the model held coverage

Pavo tried 100+ approaches across three families. Single signals barely separated from the baseline; the best 2-predicate rules matched the model on lift but covered too few trials; the Day-0 model is the one that got sharp and stayed at ~35%.

01
Single signals (1 predicate)
- IV sweeps, CART rules, engagement score
- Best single rule: about +1%
- Not enough to ship alone
02
Composite rules (2 predicate)
- Exhaustive predicate pairs, scored by IV and CART
- Per-ad-set variants, unioned to hold coverage
- Sharp, but each rule covered under target
03
Day-0 propensity model
- Logistic regression on Day-0 features
- Thresholded to the top 35%, distilled to SQL
- Sharp and at full coverage

M3 retention lift vs the 10-min watch-time baseline

Coverage

Best single signal

+1%Above target

Composite · watch-time ≥ 7m + ≥ 2m/video

+6.3%20–27%

Composite · 1 deep completion + 60s first video

+8.0%20–27%

Day-0 propensity model → SQL

+8.23%34.96%

Bars are relative M3-retention lift over baseline; coverage is the share of trials each rule fires on, against a ~35% target matched to the daily baseline cohort. Single signals cover enough but barely move retention; the composite rules get sharp but fall under target; only the model clears both.

Offline validation

Because the +8.23% is offline, the offline had to hold up

The lift comes from evaluation, not a live test yet—so the evaluation was built to earn trust. Pavo replayed the signal day by day across a full month, not on one split, checking that it was robust to the window, that coverage never broke the daily feedback ad-sets need, and that it reduced to something production could actually run.

What the replay checked	How	Result
Not a lucky window	Day-wise replay across 31 days, not a single train/test split	Evaluated on every day of the month
Coverage stays deployable	Model cohort measured against each day's baseline cohort size	Within a few percent of the baseline, every day
Reduces to production	Top-35% model distilled to one thresholded SQL rule	Reproduces the model's selection, no serving stack
Fits Meta's optimizer	7 ad-set-specific models, CAPI-ready	Built and validated per ad-set

Net: a signal stable and simple enough to activate via Conversions API—the online A/B is the last check. Absolute cohort sizes and retention rates are Seekho's and stay confidential.

The results

A production-ready signal for the cost of a week's work

What made this viable was the speed and cost of getting there: weeks of ML iteration covered inside the seven-day engagement, each idea scored in minutes rather than days.

Approaches explored by Pavo100+
Iteration loop, idea to score20–30 min
Est. annualized revenue effect~$5M

In closing

The signal changed; the system is the deliverable

In seven days Pavo absorbed Seekho's Day-0 data, validated the baseline, searched 100+ heuristics and models, built a Day-0 propensity signal per ad-set, and showed 8.23% more month-3 retention in offline evaluation, ready to confirm with an online A/B. The reusable output is the playbook behind it: pre-aggregate the Day-0 data, sweep heuristics, distill a model to SQL, emit through CAPI, and confirm with an online A/B, ready to run on the next Day-0 retention problem.

More case studies

Teams using Pavo to move the metrics that matter, in production, at scale.