Pick the metric by what a mistake does next

Metric arguments are often framed as preference. One engineer likes precision, another likes recall, someone proposes F1 as the compromise, and the threshold stays at 0.5 because that is where the library left it. None of those choices came from looking at what happens after the system is wrong.

That is the question I want a team to start with. When this output is wrong, what happens next? Does a system of record change? Does a person lose some review time? Does personal data leave the building? The answer picks the metric. Walking the threshold, one setting at a time, picks the number. A default of 0.5 picks nothing.

This essay follows module 03 of the Transition2 course, "Metrics that match the harm of mistakes." I use the module's own small examples throughout, so you can rebuild every one of them in a spreadsheet in a few minutes and check my arithmetic.

Smoke detector or car alarm: two kinds of harm

Start with two devices that do the same work. Each senses something and makes noise.

A smoke detector is tuned for recall. Burnt toast sets it off, which is annoying, but a missed fire is a catastrophe, so nobody complains that it is too sensitive. A car alarm sits at the other end. One that goes off for every passing truck gets ignored by the whole street, and an alarm everyone ignores protects nothing. Same kind of device, opposite tuning, because the two kinds of mistake do very different amounts of harm.

The module pins down the vocabulary with a fishing net. A lake holds 100 trout and 900 other fish. The net pulls up 120 fish, and 80 of them are trout. How good is the net? There are two honest answers.

• Precision asks: when the net says yes, how often is it right? 80 of 120, about 67%.
• Recall asks: of the things you wanted, how many did you get? 80 of 100, which is 80%.

A bigger net raises recall and usually lowers precision. Neither number is the "real" quality of the net. Each answers a different question, and which question matters depends on what you do with the catch.

There is a third number people reach for, and it is the one I trust least. A doctor who calls everyone healthy is right 99.9% of the time when the disease is rare, and catches nothing. Accuracy is the wrong metric whenever the thing you care about is rare, because the easy majority swamps it.

False accepts against false rejects, per action

Two terms make the harm concrete.

A false accept is a wrong answer that went through without a person seeing it. A false reject is a right answer that was sent to review anyway. The false reject wastes review time but does no damage. The false accept is the one that reaches the outside world. In most business workflows, the number you actually manage is the false-accept rate.

From there the rule is short. If the system takes an action, every false positive is damage, so precision on that action is the metric. If the system is a safety net, every miss is a breach, so recall, or its mirror the miss rate, is the metric.

The important word in that table is "action." A model does not have one correct metric. A single score can drive two actions in the same workflow, one that changes a record and one that only flags for a person, and each of those actions gets its own metric and its own threshold. When a team tells me "our model is optimized for precision," my next question is always: for which action?

Walking the threshold on five emails

A threshold is the confidence level above which the system acts. The module walks one on the smallest example that still shows everything.

A model scores five emails for whether each is a delay notice. The scores are 0.97, 0.91, 0.85, 0.72 and 0.40. Three of the emails really are delay notices: the first, the second and the fourth. The model acts only on emails scored above the threshold you choose. The example is illustrative, but five rows is enough to see the whole trade.

To reproduce it, put the five scores in column A and a 1 or 0 for "really a delay" in column B (1, 1, 0, 1, 0). Put the threshold in a single cell, say D1. In column C, mark 1 when the score is at or above D1 and 0 otherwise. Precision is the count of rows where both B and C are 1, divided by the sum of column C. Recall is that same count divided by the sum of column B, which is 3. Change D1 to 0.95, 0.90, 0.80 and 0.70 and check each row against the table.

At 0.95 the model acts once, correctly, and catches one delay in three. That is the car alarm tuned so it almost never sounds. At 0.90 it is still perfect on what it acts on and catches two of three. At 0.80 it picks up the 0.85 email, which is not a delay, and gains nothing: recall stays at 2 of 3 while precision falls from 2 of 2 to 2 of 3. You would never choose it over 0.90. At 0.70 it catches all three delays and makes one mistake. That is the smoke detector.

Notice what the default would have done. At 0.5, the model acts on exactly the same four emails as at 0.70, because only the 0.40 email sits below either line. So "we left it at the default" quietly chose the most permissive setting in the walk, without anyone asking whether a wrong delay notice is harmless or damaging.

Which row is right depends on the action. If a "delay" verdict changes delivery dates automatically, that is an action, so precision rules and 0.90 is the defensible row. If the verdict only routes an email to a person so no delay gets missed, that is a safety net, and 0.70 is the defensible row. Same model, same scores, two thresholds. The threshold is a business decision, not a property of the model.

Why F1 rarely answers a business question

F1 blends precision and recall into one score: two times precision times recall, divided by precision plus recall. For the net, that is 2 x 0.667 x 0.80 / (0.667 + 0.80) = 1.067 / 1.467, about 0.727, or roughly 73%.

That is a fine number for a leaderboard. It is almost never the right business metric, because it treats a wrong catch and a missed catch as equally bad, and in real workflows they rarely are.

Run F1 over the threshold walk and the problem shows. At 0.90, precision is 1 and recall is 2 of 3 (0.667), so F1 is 2 x 1 x 0.667 / 1.667, which is 0.80. At 0.70, precision is 0.75 and recall is 1, so F1 is 2 x 0.75 x 1 / 1.75, about 0.857. F1 prefers 0.70. That is the right answer for the safety net and the wrong answer for the action that rewrites delivery dates. The score cannot tell the two apart, because it never asked what a mistake does next.

The threshold lesson video covers the usual refinement, F-beta, where a weighting dial called beta tilts the blend toward recall (above one) or precision (below one). One weighting can favor a bolder setting and another can change the winner. That makes the problem visible without solving it. Beta states a preference. It does not tell you what risk is acceptable.

Rare events make every blended or overall score worse. The module's example is matching duplicate supplier records, where 1 pair in 10,000 is a true match. Suppose the matcher catches 95% of real matches and wrongly flags 1 in 1,000 non-matches. On a million pairs:

• True matches: 1,000,000 / 10,000 = 100. Found: 95.
• Non-matches: 999,900. Wrongly flagged: 999,900 x 0.001 = 999.9, about 1,000.
• Flags in total: about 95 + 1,000 = 1,095. Precision: 95 / 1,095, under 9%.

Accuracy on the same run reads 99.9%. A reviewer handed that queue would find fewer than 1 real duplicate in every 11 flags, while the accuracy dial says the matcher is nearly perfect. If merging is automatic, accuracy would never have warned you.

Expected harm per decision, with an error limit

Choosing the metric tells you what to watch. Choosing the threshold needs one more step: put a harm figure on each kind of outcome and add them up. I use units, because the ratio between outcomes is what matters.

The module's harm table says a wrong automatic action does 100 units of harm and a review takes 1 unit of effort. Take 1,000 decisions and two thresholds.

Per decision, that is 1,000 / 1,000 = 1 unit for the relaxed setting and 500 / 1,000 = 0.5 units for the strict one. Less automation, half the harm. Judge by automation rate and you would pick the wrong one.

The general rule: for every threshold, take the chance of a wrong action times its harm, add the chance of a review times its effort, and compare. In a spreadsheet, make one row per setting with columns for automated, wrong and reviewed, two cells holding the weights (100 and 1), and a harm column equal to wrong times the first weight plus reviewed times the second. Then swap in your own weights.

Two words from the threshold lesson make the result reportable. Coverage is the share of decisions handled automatically. Risk is the error rate among only those automated decisions. For the relaxed setting, coverage is 900 / 1,000 = 90% and risk is 9 / 900 = 1%. For the strict setting, coverage is 700 / 1,000 = 70% and risk is 2 / 700, about 0.29%. Slide the threshold and coverage and risk usually move together. Quoting coverage alone is how a team talks itself into the relaxed row.

Then the limit. The choice is not simply the lowest-harm row. It is the lowest-harm row that also keeps the error rate under a limit agreed in advance. If a setting wins on harm but breaks the limit, it is out.

One correction the lesson makes, and I think most harm tables miss it: reviewers miss things too. Charging a review at a flat 1 unit assumes the reviewer catches every error that reaches them, and they do not. The approval-rate lesson video shows why, with a dashboard where nearly every draft gets approved and many of the wrong ones get approved along with the rest. People tend to accept a suggestion instead of checking it, especially when drafts are usually fine and the queue is long. That is automation bias. So add a column for errors that slip past review and charge them at the wrong-action weight. In the lesson's story that narrows the gap, and the cautious setting still wins, but more review is not unlimited safety. Checking the checkers with independent evidence is the subject of agents need an independent exam.

So the question for the meeting is not "how much can we automate?" It is "at what setting is our total harm lowest while staying under the severe limit?" A business owner can answer that and sign it.

The severe line stays fixed

Expected harm is an average, and an average will happily trade a few severe mistakes for many small wins. Some mistakes should never be traded, such as sending a transfer to the wrong bank account.

So the lesson adds a severe-error gate: a separate limit that no average can buy its way past. In its story, a bold setting looks better on the blended score, but it goes over the severe limit, and the cautious setting stays under. The score likes the bold setting, and the gate still says no. The gate wins.

Three rules keep that gate honest.

The owner sets it before the results. Not the data team alone. The business owner decides how many severe errors they can live with before anyone sees the numbers. Otherwise the limit quietly drifts to wherever the favorite setting happened to land. I treat it like any release decision, agreed up front; the same idea runs through the release gate is the product.

One slider per type, not one per business. A routine utility statement and the first record from a new supplier are different animals. Each record type gets its own threshold, while the severe line itself stays fixed. Report precision and recall per class as well, so nobody can hide a bad category inside a good overall average.

Report risk with an upper bound. A small sample with no errors has not proved zero errors. A quick rule of thumb from the course, the rule of three, says zero errors in n checks bounds the true rate at roughly 3 / n. Zero errors in 30 checks tells you the rate is probably below about 10%. Zero in 100 gets you below about 3%. Zero in 300 gets you below about 1%. If your severe limit is tighter than what your sample can show, you have not checked enough to claim you are under it.

When someone asks how much the system can do alone, my answer follows the lesson: the threshold is ours to set, so I weigh expected harm for each record type, hold the severe-error gate fixed, and quote coverage with risk beside it.

Do the lab on a workflow you run

The module's lab takes about 30 minutes. Its first two parts are the spreadsheets above: the five-email walk, checked against the four rows, and the harm table, checked against 1,000 and 500 units.

The deliverable is the part that changes how a team works: a per-action table for one workflow you actually run. For each action, write down what a wrong yes does, what a wrong no does, the metric that follows, the harm weights, the threshold the walk supports, and the severe limit with the name of the role that owns it. If you cannot fill a row, that action is not ready to run without a person.

Both lesson videos for this module, "Set the dial by what a mistake costs" (v03) on thresholds and the severe-error gate, and "The number that looks like accuracy" (v01) on approval rates and automation bias, are in the engineering video library. Start with the threshold one.

Go deeper: the full lesson and lab are in the Transition2 course.