Incrementality testing: the channel cut that raised profit 42%
A good butcher does not trim around the bone. He cuts through it.
I did that to a games studio's ad budget. Profit went up 42%.
The setup
The quarter was soft. Everyone wanted the growth lever pulled harder: more spend, more channels, more creative.
So I pulled the numbers instead.
One line in the media plan had the best dashboard in the deck. Cheapest installs, highest ROAS, untouchable in every budget meeting.
The test
We tested it the only way that survives bad attribution: we paused it in matched regions and watched what happened to revenue.
Almost nothing happened.
The channel was not buying new players. It was re-buying players who would have installed anyway, then taking the credit. Non-incremental spend, dressed up as the star of the plan.
So I cut it. Not trimmed. Cut. Profit grew 42%.
This is not just a gaming problem
In one dataset of 225 incrementality tests run across 2024 and 2025, branded search came in at a median incremental ROAS of 0.70x. Below breakeven. A dollar in, seventy cents out, because attribution keeps giving every channel a medal for standing near the finish line.
The channels most likely to look better than they are tend to share one trait: they reach people who were already on their way. Branded search, retargeting and some app-store placements all sit close to a decision the customer had mostly made.
Why dashboards miss it
Attribution answers "who touched the conversion last?" Incrementality answers "what would have happened without this spend?" Those are different questions, and only the second one tells you whether money is working.
A channel can win the first question every day and lose the second one every day.
How to run a simple geo holdout
- Pick the channel you are most sure about. The one with the best dashboard is the best candidate, not the worst.
- Split regions into matched groups with similar size and past revenue.
- Pause the channel in the test group only. Keep everything else the same.
- Run it for two to four weeks and compare total revenue, not attributed revenue, between groups.
- If revenue barely moves, the channel is not incremental. Cut it or cut it back hard, and move the budget to what moved.
Bottom line
The cut that grows profit is rarely the obvious one. It is the fat marbled through the cut you are proud of. Looks like muscle. Feeds nothing.
Incrementality is not an audit you run once. It is the knife you keep sharp. Measure what actually breaks if you stop. Then cut what nobody would miss.
Source: branded search median 0.70x incremental ROAS from a geo-test dataset of 225 tests (August 2024 to December 2025), via Digital Applied.
FAQ
What is incrementality testing in marketing?
It measures how much revenue a channel actually causes, by comparing results with and without the spend. The most common method is a geo holdout: pause a channel in some regions and compare their revenue with matched regions where it kept running.
What is the difference between attribution and incrementality?
Attribution assigns credit for a conversion to the touchpoints before it. Incrementality asks whether the conversion would have happened without the spend. A channel can get a lot of attribution credit and still add almost nothing.
Which channels are most often non-incremental?
Channels that reach people who already intend to buy or install, such as branded search and retargeting, are common examples. In one dataset of 225 tests, branded search had a median incremental ROAS of 0.70x.
How long should a geo holdout test run?
Usually two to four weeks, long enough to cover normal weekly swings. Compare total revenue between the test and control regions, not the revenue your ad platforms attribute.