How long a geo test should run

What a week buys

A geo test measures a sustained shift in total orders across the regions that keep advertising. Its precision comes from two things: how many orders the test window contains, and how well the regions' history predicts each other. The first grows with every week. The second is fixed before the test starts. So the interval narrows quickly over the first few weeks and then flattens, because the history's noise is a floor no number of test weeks can dig under.

That is why the test is measured in weeks and not days. Day-of-week patterns and daily serial correlation add structure without adding information about a level shift; weekly totals carry the same signal with far less noise. Daily rows are still the best input, because they let the analysis cut exact weeks that start on the day the pause began.

Clean weeks, not calendar weeks

A clean week is seven complete days with the channel paused in the holdout. The day the exclusions were applied and the day they were removed are not clean: ads were live for part of them. Those days sit outside the measurement. A test "run for four weeks" on the calendar with the pause applied on a Tuesday afternoon has three clean weeks in it, and that is what gets analysed.

Why four is the default and three is sometimes enough

The smallest change a test can detect is a curve that falls as the window lengthens. For most accounts the curve between three and eight weeks is nearly flat except in a narrow band of channel shares, three to nine points wide, where eight weeks moves the answer into a better precision band. Outside that band, a three-week test answers the same question an eight-week one does. So three weeks is offered only when its precision band matches eight weeks' for that account; every other case gets four. Below three weeks the result is directional.

Shorter tests are also safer. Seasonal drift between regions accumulates with time, and so does the chance that someone changes a campaign, launches a sale in half the country, or a delivery problem hits one region. A test that answers in four weeks is a test that ran in one regime.

The detection floor by volume

These are the smallest changes in total orders a well-designed test detects at each volume, from simulations on real UK regional sales data with order counts redrawn at each scale. The floor is a property of your volume, not of the test length.

Simulated sales are cleaner than real ones, so treat these as the best case.

From orders to a rate

The floor is in total orders. The question you care about is on the incrementality scale: what share of the orders the platform credits to the channel were real. The two are linked by one division. The interval's half-width on the rate scale is the half-width in total orders divided by the orders the platform claims for the channel. At the median, a test's half-width comes in at about two thirds of the detection floor, so a channel credited with twice the floor's worth of orders comes back roughly plus or minus 33 rate points, and one credited with four times the floor about plus or minus 16.

That is the whole of the readiness rule: the channel's claimed share of your orders has to clear the floor with room to spare. Two times the floor is the line, because only part of what the platform credits is usually incremental. The readiness checker runs it for you.

The answer-by date

Working backwards from when you need the answer: apply the pause, wait for the first clean week to start, run the clean weeks, remove the pause, analyse. A four-week test started on a Monday reports five weeks later. If the date you need it by allows fewer than four clean weeks, the honest choice is a three-week test where its precision matches, and otherwise to move the date rather than the test.