A confidence score is a ranking, not a probability

The situation

Every value Halmside's extractor returns arrives with a number beside it, somewhere between 0 and 1. That number is called a confidence score. Nobody at Halmside chose what it means, and nobody was told.

The office agreed a rule in about four minutes: accept anything at 0.90 and above without looking, and queue the rest.

It is a reasonable rule. It reads as nine times out of ten this is right, and that is good enough for a delivery docket.

That is not what the number says. This lesson is about the difference, and about what the difference costs.

What the number is measured against

Naji ran the same comparison as in lesson 1, but this time he kept the score. 2,940 extracted fields over six weeks. Each one with its score, each one checked against the hand-typed value.

Overall: 2,514 of 2,940 correct, which is 85.5%, and 426 errors.

Now read the second row against the office rule. A field scoring 0.90 is right 83.1% of the time, not 90%. Halmside was auto-accepting 2,016 fields. 138 of them were wrong. That is an error rate of 6.8% inside the band everyone was calling safe.

The scores are still worth having

This is the half that gets lost when somebody discovers the problem above and throws the score away.

Sort all 2,940 fields by score, and look at where the 426 errors sit.

• The 438 lowest-scoring fields are 14.9% of the volume. They hold 173 errors, which is 40.6% of every error there is.
• Widen that to everything below 0.90: 924 fields, or 31.4% of the volume, holding 288 errors, which is 67.6% of them.

Reviewing the lowest-scoring 31.4% catches 67.6% of the errors. A review that picks fields at random catches, by definition, the same share of errors as the share of fields it looks at. So the score is worth 2.15 times a random review of the same size.

That is a real and useful property, and it has a name worth learning, because the name keeps two ideas apart in your head.

A score can discriminate well, meaning it puts the bad ones near the bottom. The same score can calibrate badly, meaning the number itself promises something it does not deliver. Halmside's scores do exactly that. They make an excellent queue and a worthless promise.

What it costs to buy the right to auto-accept

You do not have to calibrate every band. You have to calibrate the one band you are about to stop looking at.

Halmside wanted to auto-accept at 0.90 and above. To know that band's true accuracy to within five percentage points, with the usual confidence, at an accuracy somewhere near 83.1%, it needed:

216 fields, drawn from that band, checked by hand against their documents.

At 38 seconds a field, that is 2.28 hours of somebody's afternoon. That is the whole price of the right to auto-accept a band, and there is no cheaper way to buy it.

Halmside paid it. It found 83.1%, and moved its threshold to 0.95, where the measured accuracy is 97.5%. The arithmetic in lesson 7 says the errors left above that line cost less than checking for them.

The two things the score cannot see

A value that is not on the page. Of the 426 errors, 58 were fields where the document simply did not carry the value at all. No ship terms, no lab-dip reference, nothing to read. The extractor returned a value anyway.

Those 58 had a mean score of 0.91, above the sample mean of 0.89. It was confident because it was confident about what it produced, and nothing in the score asks whether the thing existed in the first place. Course 10.1 lesson 2 already prices what an invented value does to the decisions below it. The addition here is that an invented value tends to score high, so a threshold rule waves it straight through.

A document it has not seen before. The calibration you just paid for holds only while your documents stay as they are. When one supplier changes a template, the scores stay where they were and the accuracy underneath them moves. That is lesson 6, and it is the failure that arrives with no announcement.

Act as a measurement analyst helping a garment factory decide whether it may stop reviewing some of its extracted document fields. I have run a hand check and I have the raw results. My data: [PASTE ONE ROW PER SCORE BAND — THE BAND, HOW MANY FIELDS FELL IN IT, AND HOW MANY OF THOSE WERE CORRECT WHEN CHECKED BY HAND]. My review takes about [SECONDS] a field and my loaded office rate is [AMOUNT] an hour. The band I want to stop reviewing is [BAND]. Do the following. First, give me the measured accuracy of every band and say plainly, band by band, whether the score behaves like a probability of being right. Second, tell me how many fields I would need to hand-check in the band I named to know its true accuracy to within five percentage points, and convert that into hours and money at my rate. Third, tell me separately how well the scores ORDER my fields: if I reviewed the lowest-scoring fields first, what share of all my errors would I have caught by the time I had reviewed a tenth, a quarter and a third of the volume, and how does that compare with reviewing the same number at random. Fourth, state which of those two properties my scores have and which they do not. Fifth, list what would invalidate this measurement — a new supplier, a changed template, a new document type — and tell me how I would notice. Show the arithmetic for every figure so I can check it, and do not fill any gap in my data with an assumed number.

Lesson 4 leaves extraction and takes up the second job: deciding that two records which nearly agree are about the same thing.