Lessons · Lesson 5 of 5
Every number in it was right
Audit a decision page for comparability rather than for accuracy, and attach to each figure the four facts that say whether it can sit beside the one next to it.
Lesson 5 of 5 · 20 min
The page that moved two lines
The quarterly board pack at Velanthur runs to eleven pages. The one that mattered was page four. It was four bullets long, and Kemal Gubran acted on it inside a week.
| What it said | Is it true? |
|---|---|
| On-time delivery: Norrhavn 97.2%, Palliser 71.4% | Yes |
| Palliser's on-time delivery has fallen from 84.6% to 71.4% | Yes |
| Palliser's order book for next season is down 18.2% | Yes |
| Average FOB: Palliser USD 6.85, Norrhavn USD 9.40 | Yes |
The conclusion written underneath was: Norrhavn is our best-performing account and Palliser is deteriorating; capacity should move. Kemal moved two of fourteen sewing lines from Palliser work to Norrhavn work for two quarters.
Nothing on the page is false. Nobody rounded anything in their own favour. An auditor recomputing all four figures from the underlying records would confirm all four.
Statement one: two figures, two definitions
Lesson 1 counted thirty-six honest ways to compute on-time delivery from one ledger. Both figures on page four are on that list, and they are not the same entry.
Norrhavn's 97.2% comes from Norrhavn's supplier portal. That portal measures against the last revised date, counts the first shipment, and takes the population as shipments made. Palliser's 71.4% comes from Palliser's letter: original confirmed date, in full, over lines due. Nadira recomputed both accounts on each definition.
| Definition | Palliser | Norrhavn |
|---|---|---|
| Portal basis: revised date, first shipment | 96.3% | 97.2% |
| Strict basis: original date, in full, over lines due | 71.4% | 67.5% |
On the portal definition, Norrhavn is ahead by 0.9 points. On the strict definition, Palliser is ahead by 3.9 points. The 25.8-point gap the board acted on exists on neither definition, and on no definition at all. It is the distance between two different questions.
Norrhavn's account looks excellent on its own portal, for a reason that is not delivery performance. Norrhavn revises dates freely and asks for part shipments as a matter of course. A measure that forgives both is measuring a buyer's habits rather than a supplier's punctuality.
Statement two: the base quarter was the unusual one
Palliser's on-time delivery fell from 84.6% to 71.4%, a drop of 13.2 points. Both figures are correct on Velanthur's own convention.
That convention, from lesson 1, drops cancelled lines out of the denominator. In the base quarter, Palliser had run a range review and cancelled 18 of 96 lines. Nine of those eighteen were cancelled after Velanthur warned it would be late. That is the survivorship case from lesson 1, sitting in the quarter being used as the baseline.
Put those nine back and the base quarter is 66 of 87, or 75.9%. Against the current quarter's 71.4%, the fall is 4.4 points.
Two-thirds of the reported deterioration is a property of the quarter it is being compared against. A percentage change is a statement about two numbers, and only one of them is this quarter's. A report that says a figure moved has quietly claimed that the other end of the comparison was ordinary, and nobody ever checks that end.
Statement three: the number contains the date it was run
Palliser's order book for next season was down 18.2% on the same point last year, as at the day the pack was produced.
Palliser places orders in two waves: a core wave, and a seasonal wave five weeks later. Norrhavn places once. So a report run on a fixed calendar date catches all of Norrhavn's intake and only part of Palliser's. The share it catches depends on nothing but which day somebody pressed the button.
Cut both books at the same number of weeks after each buyer's own season opening, and Palliser is down 3.1%.
Statement four: true, and about the wrong quantity
Palliser's average FOB is USD 6.85 and Norrhavn's is USD 9.40. That is correct, and it is not evidence for the conclusion. A capacity decision does not turn on the price of a garment. It turns on what a line-day earns, and Norrhavn's more expensive garment takes longer to make.
Track 14 owns that ranking and works it out properly. The point here is only this: page four contained a price and no line-day figure, and the conclusion it reached was a capacity conclusion. The most dangerous number on a page is often a correct one that answers a question next door to the one being decided.
What the decision cost
Norrhavn could not absorb the capacity. Its order book was conservative, which is part of why its portal figure looked so good. So the two transferred lines ran at 68.4% booked over two quarters, against the 96.1% the Palliser work had been running at.
Two lines, 128 line-days each: 70.9 idle line-days at a contribution of USD 388.20 — USD 27,523.38.
That is the part that can be proved. Palliser also reduced Velanthur's allocation at the next range review, and its buyer mentioned capacity when she did. Nobody can demonstrate what would have happened otherwise, so it is not in the number. A reviewer should attack the temptation to add it, rather than the figure above.
The repair: four facts under every number
Velanthur's page four now carries a provenance line under each figure, in small type. Four facts, and the rule is that a number without them cannot go on a decision page.
| Fact | What it prevents |
|---|---|
| Population — what set the figure is over | Two figures over different sets read as a comparison |
| Basis — the definition, in the words the buyer or the system uses | Statement one |
| Period — the dates, and how a transaction enters them | Lesson 4's whole subject, on a single line |
| Run date — the day the query was executed | Statement three |
Rewritten with provenance, page four's four bullets support one conclusion: Palliser's delivery performance is broadly flat, Norrhavn is not measurably better on any common definition, both order books are roughly where they were, and the capacity question needs a contribution-per-line-day figure that this page does not contain.
That is a duller page, and it would not have moved two lines.
Prompt · Audit a page for comparability
Before a management page is presented, especially one whose numbers have all been checked and are all correct.
Act as a reviewer whose only job is to decide whether the numbers on a page can sit beside each other. Assume every number is arithmetically correct, and do not recompute any of them. Here is the page, exactly as it will be shown, including its title and any conclusion written under it: [PASTE IT]. For each number I can also tell you where it came from: [FOR EACH ONE - WHICH SYSTEM OR PERSON PRODUCED IT, WHAT SET IT IS COMPUTED OVER, WHAT DEFINITION IT USES, WHAT PERIOD IT COVERS AND HOW A TRANSACTION ENTERS THAT PERIOD, AND THE DAY THE QUERY WAS RUN]. Say clearly which of those facts I have failed to give you, rather than guessing them. Do the following. First, build a table with one row per number and one column per provenance fact, and mark every cell I could not fill. Second, find every pair of numbers on the page that a reader would naturally compare, and for each pair say whether their provenance facts match. Third, for every pair that does not match, say what the mismatch is worth if you can estimate it from what I gave you. If you cannot, say what single piece of data would let you. Fourth, look at any figure expressed as a change over time, and ask what was unusual about the earlier period, naming what you would need to check. Fifth, check the precision: for each number, say what the weakest input behind it supports, and whether the page claims more than that. Sixth, take the conclusion written under the page and tell me which of it survives, which of it depends on a mismatched comparison, and what question the page would need to answer to support the rest. Seventh, rewrite the page's conclusion using only what survives. Do not soften it, and do not add a recommendation of your own.
AI can make mistakes — check anything you act on.
Check yourselfA page compares your performance for two buyers, and both figures came out of the buyers' own portals. What is the fastest check?Show the answer
Recompute one buyer's figure using the other buyer's definition. It usually takes an afternoon, on data you already hold, and it settles the question completely. Either the ranking survives both definitions, in which case it is real, or it does not, in which case the page is comparing two questions and the gap on it belongs to neither buyer.