Two numbers, and the one everybody trusts

The month the two systems disagreed

At the end of the pilot's first month, Nermine Sadek, who runs Elmarj's payroll, brought Ramez a sheet of paper. Her production register for Line 4 said 41,088 pieces. The tracking system said 39,412.

The gap is 1,676 pieces, 4.08%. It is not a rounding difference and it is not noise. Somebody has to be wrong.

The meeting that follows is the same in every factory. The system is new, the register is old, and the register is what wages were paid from last month, and the month before, and for eleven years. So the tracking figure is put aside "until it settles down". It never settles down. Eighteen months later the plant is running a system whose numbers nobody quotes.

Ramez did the other thing. He reconciled the two line by line until the gap was fully explained. It took a day.

Four causes, and only one of them is a fault

Repair passes, 604. A garment sent back for a broken stitch goes through the operation a second time. The operator is paid for the second pass, and rightly: she did the work. The tracking system counts the garment once, and rightly: one garment was made. Both numbers are correct answers to different questions. How much work was done, and how many garments exist. There is no reconciliation to perform here, only a definition to write down.

Saturday overtime, 512. The terminals sit on a network switch, which is on the office battery backup, and the office is powered down at weekends. Nobody decided this. It came out of two entirely sensible decisions taken by two departments who never spoke to each other. The payroll figure is right. The tracking figure has a hole in it that repeats every Saturday, invisibly, because nothing on the screen says a day is missing.

Rejects never repaired, 248. The tracking system records the garment as having passed operation 14, which it did. Payroll pays the operator for it, which is also correct. It is never shipped. Both figures are right about their own question, and neither is a shippable-goods figure. That was the question the sales meeting was actually asking when it quoted one of them.

And then the fault, 312. When a bundle is topped up after a recut, the ticket sometimes carries 22 pieces instead of 20. The quantity field on the terminal is optional, and when it is left blank the system writes the standard bundle size. 156 bundles that month carried 22 pieces and were recorded as 20.

That is not a definition problem. It is a system silently inventing data.

Why payroll wins, and why it deserves to

Nermine's register is not more accurate because it is older, or because payroll is more careful. It is more accurate for one structural reason: a number somebody is paid from is checked every month by the person who is paid it.

Three operators queried their pay slips within a day of receiving them that month, and all three were right. In fourteen weeks not one person queried a figure on the tracking screen. Nobody's pay depends on that screen. So nobody goes looking for mistakes in it, so its errors build up quietly, while payroll's errors are found by the best-motivated checkers in the building.

This has a design consequence that is worth more than the observation. If you want a production number to be right, give it to somebody whose money depends on it, and let them see it. Elmarj now prints the recorded pieces on the pay slip beside the paid pieces, with the difference and its reason. The number stopped being wrong within two months, and it did not stop because anybody fixed the software.

What to do with the two numbers

Do not reconcile them monthly. A monthly reconciliation is a standing admission that the definitions were never written down.

Print the definition beside the number. Elmarj's screens now carry one line under each count:

Garments completed at this operation. Repairs counted once. Rejects included. Excludes any hour the terminals were not powered.

Three sentences, and the argument in the September meeting could not have happened. The last of them was added because of the Saturday hole, and it is the most valuable of the three: a count must say what it leaves out, not only what it includes. Otherwise an absence looks exactly like a zero.

Act as a factory controller who has reconciled production counts against wage records many times, and knows that most of the gap is definitions rather than errors. Facts: period [MONTH], line or lines [WHICH], tracking system figure [PIECES], payroll or production register figure [PIECES], how the incentive is calculated [DESCRIBE], bundle size [PIECES], what happens to a repair pass [DESCRIBE], what happens to an end-line reject [DESCRIBE], any overtime or shifts worked outside normal hours [DESCRIBE], and any fields on the recording screen that are optional [LIST THEM]. Do the following. First, list every mechanism that could produce a difference between these two counts, and for each one say which of the two systems is right, or whether both are right about different questions. Second, help me design the check that would size each mechanism from records I already hold, and put them in the order I should test them, cheapest first. Third, and most important, tell me which candidates are DEFINITION differences and which would be real faults. Treat any optional field with a default value as a fault until proved otherwise: a blank forced to a plausible number is a false record that looks exactly like a true one. Fourth, draft the definition line, of no more than three sentences, that should be printed under each count on every screen and report, stating what is counted, when it is counted and what is LEFT OUT. Fifth, tell me what to do so that the tracking number gets somebody checking it: who should be shown it, beside what, and why a number nobody is paid from stays wrong indefinitely. Ask me for anything you need that I have not given you, rather than assuming it.