The failure that arrives with no version number

The situation

On 4 May, Pellworm Mills issued its invoices on a redesigned template. The purchase-order number moved from a block at the top to a line in the footer, and a second column was added for the mill's own currency.

Nobody told Halmside, and there was no reason to. It is Pellworm's invoice.

No software was released. No setting was changed. No score moved. The extractor was as confident on 5 May as it had been on 3 May, for the reason lesson 3 gives: a confidence score describes the system's own certainty, and it knows nothing about whether it has seen this layout before.

This is a failure with no version number attached to it.

Why it is a different problem from the one you know

Course 24.2 lesson 6 has already done the general arithmetic of a count that drifts. Why an error that always leans one way piles up while a random error cancels out. Why a growing gap hides inside a normal amount of work in progress. Why four hand counts settle what one cannot. None of that is repeated here, and all of it applies to this case unchanged.

What is different is when the drift starts, and whose calendar the event is in.

A machine's cycle time moves when the garment changes: a longer body, a new size ratio, a different seam. That is your event. It happens in your factory, on a date you know, and course 24.2 puts the recalculation on the style hand-over checklist beside the standard minutes, where a person will see it.

A document reader's error rate moves when somebody else's template changes. There is no hand-over. There is no checklist to put it on. Your supplier's marketing department redesigned a form, your matching accuracy fell by a factor of eight, and every party to that sentence behaved correctly.

Would a monthly check have caught it?

Halmside's first instinct was a sampling plan: 20 invoices a month, drawn across all six suppliers, checked by hand. 240 invoices a year at four minutes each is 16 hours, or USD 182.40 at the office rate.

Now look at what that plan actually watches. Pellworm sends 12.9% of the invoices, so a proportional sample of 20 contains about 2.6 of them a month. You cannot conclude anything from two or three documents. So the plan spends sixteen hours a year watching everything except the supplier carrying 61.8% of the money.

Suppose instead that every Pellworm invoice were checked. That is 82 over the six months after the change, about 14 a month. At the new rate of 14.6%, the chance of seeing two or more errors in a month is 62.8%. Two months takes it to 86.1%, and three months to 94.8%.

Two months is a respectable detection time for a rate. It is a terrible one for this factory. Two months at 14.6% on 28 invoices is four errors that have already happened, and lesson 4 measured what a matching error costs once something acts on it.

The rate signal arrives after the money. So stop trying to measure a rate.

Where the money is concentrated, do not sample. Count

Checking every Pellworm invoice is 148 a year at four minutes: 9.9 hours, or USD 112.48.

That is 38% less than the proportional sampling plan, and it is not the same kind of control at all. At full coverage you are no longer estimating a rate. Every error is found on the invoice it is on, before it settles a line, and the step on 4 May shows up as three wrong matches in the first fortnight instead of as a number in a monthly table.

What the check has to be checked against

One more requirement, and it is the one most often got wrong.

Halmside's three-way match compares the purchase-order line, the goods-received docket and the supplier invoice. It works as a check because those three records have three independent origins. The docket is a hand count made at the gate. The invoice is the supplier's claim. The purchase-order line is Halmside's own. Three people wrote three numbers without seeing each other's.

Now take a "reconciliation" that compares an extracted value against the same PDF it was extracted from. That has measured nothing at all. It will agree with itself every time, and it will agree with itself most enthusiastically in exactly the case lesson 2 describes, where the value is a perfect reading of the wrong document.

Course 24.2 explains why a hand count, with real errors of its own, is still the right instrument against a systematic one. The addition here is the harder half: the second source must not come from the first. Ask of any check you are offered: if the machine read the document wrongly, would this check read it wrongly too? If the answer is yes, it is a copy, not a check.

Lesson 7 takes everything measured so far and turns it into one decision, made six times.