60 tests. 115% CVR. $1.2M over goal.
Inherited a funnel that hadn't been touched with real product rigor. In year one: ran 60+ A/B tests, drove a cumulative 115% CVR improvement, and delivered $1.2M over revenue goal on a ~$19M DTC funnel.
+115% CVR
$1.2M over goal
+11.1% new subscribers
# The situation
The Sunday acquisition funnel had years of instinct behind it and no way to prove any of it. When CVR dropped, the team could list what shipped. Nobody could explain what caused it. I inherited it in April 2023 — no experimentation culture, no clear measurement standard, no owner. My mandate was vague: fix it.
Six months later, our CPO and my direct manager were laid off. I had no air cover and a new VP of Engineering who didn't yet trust the process I was proposing. That's when I made the bet: we'd go faster if we slowed down.
The system I built
Before I ran a single test, I built the infrastructure to learn from them.
A PM control room — Discovery, Delivery, Impact, and AB Testing views — gave the team real-time visibility into where every bet was in its lifecycle. A weekly meeting I called Thunderdome brought together senior stakeholders and the data team to pressure-test every test before it launched: was the hypothesis sound? Were the success metrics measuring the right thing? Were guardrails in place?
Every test was documented to a standard: background, hypothesis, control vs. variant experience, primary success metric, secondary metrics and guardrails, learnings, next steps. Jira linked discovery tickets to engineering delivery tickets to AB test cards — so nothing lived only in my head.
I didn't rely on VWO's binary winner/loser calls. I built custom Heap dashboards to measure clicks, scrolls, plan purchases, and behavior funnels. I used Omni to check AOV impacts. When Session-Based CVR turned out to be measuring marketing efficiency rather than funnel health, I invented a new metric — Qualified CVR — to cut through the noise.
Then I trained a Jr. PM to run it.
The steerco
At the end of the season, I led a cross-functional executive presentation on the most contested strategic question: how to balance CVR optimization against retention, margins, AI innovation, and reduced engineering bandwidth for 2026. Pulled in data analysis, supply chain risk, EBITDA scenarios, retention-vs-acquisition tradeoffs.
The President's feedback: "Best steerco in the history of our steercos. This is exactly what this meeting is for."
Results
Cumulative 115% CVR improvement. $1.2M over revenue goal on a ~$19M DTC funnel. New subscribers up 11.1% YoY. Off-peak CVR up 121%.