Seven true answers to one question

Two numbers, no liars

On a Tuesday in October, Nadira Sallehin is reading two pieces of paper about the same three months. She is head of merchandising at Velanthur Apparel.

The whiteboard behind her desk says on-time delivery, Palliser Stores, quarter three: 96.4%. It has said something in the nineties all year.

The letter in front of her comes from Marta Ocklund, who runs vendor performance at Palliser. It says Velanthur's on-time delivery for the quarter was 71.4%. That puts the account in the bottom band of Palliser's supplier list. A recovery plan is expected within twenty working days.

Neither number is wrong. Nobody has miscounted anything. There is one list of shipments, and both sides agree on every date in it. The two figures are twenty-five points apart.

The ledger everybody agrees on

Palliser bought against 28 purchase orders in the quarter. One purchase order carries several delivery lines. A delivery line is one colour, one size curve, one destination, and its own ex-factory date — the day the goods must leave the factory. The 28 orders carried 84 lines between them.

Nobody disputes any of that. Marta Ocklund's system holds the same 84 rows. What differs is the arithmetic done on them. And the arithmetic is not really a calculation. It is four decisions, each made by a different person, for a different reason, mostly years ago.

The four decisions

The unit. Do you count purchase orders, delivery lines, or pieces? Velanthur's report counts purchase orders, because the report started life as a list of orders. Palliser counts lines, because a line is what a store gets.

The date basis. On time against which date? The one Palliser asked for, the one Velanthur first confirmed, or the last revision the two of them agreed?

The completion rule. Does a part shipment on the date count as on time, or must the line be complete? "On time" and "on time and in full" are different questions, and the second one is harder.

The population. Over what set? Everything due in the quarter, or everything shipped in the quarter? The two differ by the cancelled lines, and by anything still open at the quarter end.

Three units, three date bases, two completion rules, two populations. That gives 36 different figures, all computed correctly, from one ledger nobody disputes. Seven of them were in use inside the two companies that October.

The whiteboard is the first row: 27 of 28 orders hit their last agreed date, counting the first shipment. The letter is the sixth: 60 of 84 lines hit the date Velanthur first confirmed, complete, including the ones Palliser cancelled.

What each choice is worth

The spread is not the useful part. The useful part is that the four choices are worth very different amounts, and the two worth most are the two nobody writes down.

The date basis is the whole argument. Everything else together moves the figure by less than eight points. The date basis moves it by seventeen. And the date basis is the one choice that never gets printed on a report. Every system has a field called something like ex-factory date, and it holds whatever was written into it last.

The unit looks harmless here. It is worth 0.1 of a point between orders and lines, and that is an accident of this quarter. Count in pieces and the figure jumps to 81.7%, ten points above the line figure, because Velanthur's late lines were small ones. The 21 live lines that missed their date carried 64,800 pieces between them. The 60 that hit carried 336,700. A piece-weighted figure and a line-weighted figure disagree by about ten points whenever the small orders are the difficult ones, which is most of the time.

The one choice that cannot be defended

Three lines were cancelled by Palliser and dropped out of Velanthur's denominator. Nadira went and read why they were cancelled.

Two of the three were cancelled because Velanthur had written to say it would miss the date by three weeks, and Palliser withdrew the drop rather than take it late. Those are not lines that left the population. They are the worst results of the quarter, removed from the measurement for being the worst.

That is what separates this choice from the other three. Counting orders instead of lines is a choice about grain. Counting the revised date is a choice about what the number is for. Dropping the cancellations is a rule that removes you from the sample when you fail badly enough. No version of it survives being said out loud. Velanthur's own convention was arguing that the more damage it does to a buyer, the better it scores.

What Nadira actually did

She did not pick the truest number, because there is no such thing. She did three things.

1. Adopted Palliser's definition for Palliser. Lines, original confirmed date, in full, over lines due. Not because it is more honest than the others, but because it is the one the account is scored on. A supplier that reports itself on a definition its buyer does not use is reporting a number nobody can discuss.
2. Kept exactly one internal figure, on a definition written out in a sentence: lines shipped complete on or before the date Velanthur confirmed at order acceptance, over all lines with a confirmed date falling in the period, including lines cancelled after acceptance. Twenty-nine words a stranger can follow without asking anybody a question.
3. Put the cancelled lines back, in both figures, for good.

The internal number went from 96.4% to 71.4% in one afternoon. Kemal Gubran, the managing director, asked whether performance had collapsed. It had not moved at all. That conversation is the price of the correction, and you pay it once.

Course 10.1 owns the field this figure is built from: who owns a date, and when a blank in it is honest. This lesson is about the level above that, where those fields get counted and the count goes on a page. Course 24.3 asks the same question on a shop-floor screen, where the reader is a supervisor with an hour to act in. Here the reader is a buyer with a supplier list to sort.

Act as a supplier-performance analyst who has reconciled buyer and factory scorecards and has no interest in making either side look good. I want the definition behind one of my figures written out, and every honest alternative to it computed. The figure is [NAME IT, e.g. on-time delivery]. I report it as [VALUE] for [PERIOD]. My buyer reports [VALUE] for the same period. Here is my ledger for the period, one row per delivery line: [PASTE IT, WITH FOR EACH LINE - THE ORDER NUMBER, THE QUANTITY, THE DATE THE BUYER REQUESTED, THE DATE I FIRST CONFIRMED, THE LAST REVISED DATE IF ANY, THE DATE THE GOODS ACTUALLY LEFT, THE QUANTITY ACTUALLY SHIPPED ON THAT DATE, AND WHETHER THE LINE WAS CANCELLED AND BY WHOM]. Do the following. First, state the four choices my current figure makes: the unit it counts in, the date it measures against, whether a part shipment counts, and what set it is computed over. If you cannot tell one of them from what I gave you, say which and ask. Second, compute the figure for every combination of those four choices. Put them in one table, sorted from highest to lowest, showing the count and the denominator for each. Third, tell me the spread between the highest and the lowest, and how many points each choice is worth when I move it on its own. Fourth, say which combination my buyer is most likely using, and what evidence in my ledger points to it. Fifth, look at the cancelled lines one by one. For each, say whether it was cancelled for a reason connected to my own lateness. If any were, show the figure with them put back. Sixth, write my chosen definition as a single sentence that somebody outside my company could follow against the same ledger without asking a question, and tell me which words in it are still unclear. Do not tell me which figure is the true one, and do not average the alternatives.