A contract negotiation earlier this year turned on a number nobody wanted to say out loud.
The goal, as first written, was to take activation from 5.1% to 10% — roughly double it. It reads like an ambitious, well-specified target. Then somebody did the arithmetic. The account did about 250 signups a month. Doubling that rate meant moving about twelve users.
Twelve. Over a quarter of work, with a real fee attached, and the entire measurable outcome was twelve people. Worse, a more modest version of the same goal — a one- or two-point improvement, the kind most teams would call a solid quarter — came to two or three users. Two or three.
At that volume a percentage is not a measurement. It’ a rounding error wearing a suit.
What a rate actually requires to mean something
A conversion rate is a ratio, and a ratio only carries information when the denominator is big enough that the numerator can’t swing on luck. That’ the whole trick. Nothing about it is sophisticated.
At 250 signups a month, a single unusually good week — one enterprise-shaped account that signs up five users, one customer who happened to have their data ready — moves the rate by a full point. So does the opposite: one holiday week, one outage, one big account that stalls. The natural month-to-month wobble of the number is roughly the same size as the improvement you’re trying to produce. Your signal and your noise are the same height.
This produces two failures, and they arrive in order. First, you can’t tell whether the work did anything. Second — and this is the expensive one — you can’t tell whether it “didn’t” either. A team that ships a genuinely good activation change into a low-volume funnel and reads a flat rate will conclude the change failed and revert it. The change was fine. The instrument couldn’t see it.
Like this? I send one idea like it most weeks. One PLG idea a week → subscribe
Below the line, measure in people
The fix is not a better statistical method. It’ a different unit.
Below roughly a thousand signups a month — and most companies doing this work seriously are well below it — the honest success metric is an absolute count inside a fixed window. Not “raise activation from 5.1% to 10%.” Instead: “twelve activated accounts in a thirty-day window.”
Same ambition. Same underlying work. Completely different relationship with reality. Twelve accounts is a thing you can name. You can list them. You can open each one and look at what it did, which is the part that actually teaches you something. Nobody has ever learned anything from 4.8%%.
A count also forces a definition. To count twelve activated accounts you have to say what an activated account “is”, which means naming the specific event — a completed Golden Path step, not a proxy like “logged in twice.” Rates let you stay vague, because the denominator does the talking. Counts don’t.
Then put the window at the end
The second half of the rule is about “when” you measure, and it is the half that gets negotiated away.
The measurement window belongs at the “end” of the engagement, not across all of it. A ninety-day engagement that scores itself on all ninety days is grading its own setup phase — the weeks spent instrumenting events, finding the First Value Moment, shipping the first version of a change and then fixing what the first version got wrong. That work is necessary and it produces almost no conversion. Averaging it in guarantees the final number understates whatever was built.
So: build for sixty days, measure the last thirty. The number you report is the number the system produces once it’ actually running, which is also the only number that predicts anything about next quarter.
This is not a scoring trick in the practitioner’ favor. It cuts the other way just as hard — it removes every excuse about ramp-up, and it means the final thirty days have to stand on their own with nothing to hide behind.
This is the Delta-Only Attribution Model doing its job
All of this is one idea wearing different clothes. The Delta-Only Attribution Model says you get credit for the change against a documented day-one baseline — not against a benchmark, not against an industry average, not against a number that sounded good in a pitch.
An absolute count in a fixed window is what that model looks like when volume is low. The baseline is a count too: “in the thirty days before we started, this account activated three times.” The target is a count: twelve. The delta is nine real accounts, each of which has a name and a history you can inspect. There is no room to argue about relative versus absolute, no room to quietly redefine the denominator halfway through, and no way to claim a win that consists of three users and a favorable week.
It also protects against the failure in the other direction: a flattering rate that hides a small number. This is the same disease as reading a conversion rate without the gate that produced it — the ratio looks respectable while the thing underneath it is tiny or filtered. Counting forces the actual size of the business into the open.
Where the line is, and what to do on either side of it
There’ no exact threshold, and pretending otherwise would be its own kind of dishonesty. The practical test is this: compute what a one-percentage-point move is worth in users. If the answer is small enough to count on your hands, you are below the line.
- Below the line. Set an absolute count in a fixed window at the end of the period. Document the equivalent count for the baseline window. Name the activation event precisely. Expect to review the individual accounts, because at this volume the qualitative read is the better data.
- Above the line. Rates are fine, and they’ the right unit for comparison across time and cohorts. Still document the day-one baseline before anything ships, and still know which of your numbers actually drives the others rather than watching all of them at once.
The uncomfortable version of this, for anyone selling conversion work: a low-volume account cannot honestly be promised a percentage. The correct move is to say so during the negotiation, convert the goal into people, and let the smaller, truer number sit on the table. It is a worse-looking commitment and a much better one to be held to.
Twelve accounts you can name beats a percentage point nobody can defend.
Every engagement starts by documenting the day-one baseline in whatever unit the volume can actually support. That’s how the Trial Conversion Engine measures itself.
Running a trial you’re not happy with? Book a call and I’ll walk your funnel live.
