12.3.2 — Measuring a Classifier

A data scientist at Cartwise presents a new fraud model with one number on the slide: 99.5 percent accuracy. The room is impressed. Then somebody asks what fraction of orders are fraud. The answer is 0.5 percent. Which means a "model" that says genuine for every single order, catching no fraud at all, also scores 99.5 percent accuracy.

The maths was correct. The number was useless. When one outcome is rare, accuracy (the fraction of predictions that are right) is dominated by the common outcome and says almost nothing about the rare one, which is usually the one you care about. This page builds the measurements that do work: the confusion matrix, precision and recall, the ROC and precision-recall curves, choosing a threshold from money rather than habit, and calibration. Along the way it settles the reviewers' complaint from 12.2.5, where a model with excellent-sounding numbers sent them mostly genuine orders.

We will use one small, concrete test set throughout: twelve orders the model has never seen, each with the model's fraud probability and the truth.

Order	1	2	3	4	5	6	7	8	9	10	11	12
Score	0.95	0.88	0.81	0.72	0.64	0.55	0.47	0.38	0.30	0.22	0.15	0.08
Fraud?	yes	yes	no	yes	no	no	yes	no	no	no	yes	no

Five of the twelve are fraud. (Real test sets have far more orders and far less fraud; this one is small so every number can be checked by hand.)

1. The confusion matrix: four kinds of outcome

Pick a threshold, say 0.5, and flag every order scoring at or above it. Each order now lands in one of four boxes, depending on what the model said and what was true.

The confusion matrix at a threshold of 0.5. Green and blue are correct decisions; red and amber are the two different kinds of mistake, which almost never cost the same.

This grid is called the confusion matrix, because it shows exactly where the model gets confused. The names follow one pattern, and once you see the pattern you never have to memorise them. The second word says what the model did: positive means it flagged the order, negative means it passed it. The first word says whether that was right: true or false.

• A true positive (TP) is fraud the model flagged. We have 3: orders 1, 2 and 4.
• A false negative (FN) is fraud the model passed. 2 of them: orders 7 and 11. Cartwise ships these and loses the goods.
• A false positive (FP) is a genuine order the model flagged. 3 of them: orders 3, 5 and 6. Real customers get blocked or delayed.
• A true negative (TN) is a genuine order passed. 4 of them.

Accuracy is the diagonal, (3 + 4)/12 \approx 0.58. But the whole point of the grid is that the two off-diagonal boxes are different mistakes with different costs, and accuracy adds them together as if they were the same.

2. Precision and recall: two questions, two numbers

Two questions matter for a fraud model, and each one gets its own number.

"When the model flags an order, how often is it really fraud?" That is precision:

\text{precision} = \frac{TP}{TP + FP} = \frac{3}{3 + 3} = 0.50

It looks only at the flagged column. It is the number the human reviewers live with: half the orders they are sent are genuine. In 12.2.5, the reviewers' 18 percent was exactly this number, and its lowness came from the rarity of fraud.

"Of all the real fraud, how much did the model catch?" That is recall:

\text{recall} = \frac{TP}{TP + FN} = \frac{3}{3 + 2} = 0.60

It looks only at the fraud row. It is the number the finance team lives with: 40 percent of fraud still gets through. Recall is also called sensitivity or the true positive rate, especially in medicine, where it means "of the people who have the disease, how many did the test detect?"

These two pull against each other, and you can feel why. Lower the threshold and you flag more orders: you catch more fraud (recall up) but you also flag more genuine customers (precision down). Raise the threshold and the opposite happens. Every classifier lives somewhere on that trade-off, and the threshold is the dial that moves it. So whenever somebody quotes you a precision or a recall, the right question is "at what threshold?"

Here is how the dial moves on our twelve orders:

Threshold	Flagged	TP	FP	FN	Precision	Recall
0.90	1	1	0	4	1.00	0.20
0.70	4	3	1	2	0.75	0.60
0.50	6	3	3	2	0.50	0.60
0.40	7	4	3	1	0.57	0.80
0.10	11	5	6	0	0.45	1.00

At 0.90, the model is very sure of the one order it flags, but it misses four out of five frauds. At 0.10, it catches every fraud but flags almost everything. Neither end is useful on its own, which is why a single number is never enough to describe a classifier.

F1: one number when you must have one

Sometimes you need a single number, for example to compare a dozen models quickly. The standard choice is the F1 score, which combines the two:

F_1 = \frac{2 \times \text{precision} \times \text{recall}}{\text{precision} + \text{recall}} = \frac{2 \times 0.5 \times 0.6}{0.5 + 0.6} = \frac{0.6}{1.1} \approx 0.545

This is the harmonic mean of precision and recall, a kind of average that is dragged down hard by whichever of the two is smaller. A model with precision 1.0 and recall 0.01 has a plain average of about 0.5, which sounds respectable, but its F1 is about 0.02, which correctly says it is useless. That is why F1 uses this kind of average. Its weakness is that it treats precision and recall as equally important, which for fraud they are not, so use it for quick comparisons and use cost (section 4) for the real decision.

3. Curves: judging a model across every threshold

Since every threshold gives a different precision and recall, a fairer way to judge a model is to look at all thresholds at once. Two curves do this.

Two views of the same model across every threshold. Left, the ROC curve: further towards the top-left corner is better, and the dashed diagonal is a model that guesses at random. Right, the precision–recall curve: further towards the top-right is better, and the dashed line is the precision you would get by flagging everything.

The ROC curve and AUC

The left panel is the ROC curve. It plots recall on the vertical axis against the false positive rate on the horizontal axis, where the false positive rate is the fraction of genuine orders that get flagged, FP / (FP + TN). To draw it, start with the threshold above every score (nothing flagged, bottom-left corner) and lower it one order at a time. Each fraud you pass moves the line up one step; each genuine order you pass moves it right one step. Our curve goes up twice (orders 1 and 2 are fraud), then right (order 3 is genuine), then up (order 4), and so on until every order is flagged at the top-right corner.

A perfect model goes straight up the left edge, catching all fraud before flagging any genuine order. A model that guesses at random walks along the dashed diagonal. The name is a leftover from the Second World War: it stands for receiver operating characteristic, and it was developed to measure how well radar operators could tell enemy aircraft from noise on their screens at different sensitivity settings.

The area under this curve, the AUC, summarises it in one number from 0.5 (random) to 1.0 (perfect). It also has a lovely plain meaning that is worth remembering: AUC is the probability that a randomly chosen fraud gets a higher score than a randomly chosen genuine order. You can check that by counting. There are 5 frauds and 7 genuine orders, so 5 \times 7 = 35 fraud-versus-genuine pairs. Order 1 (0.95) beats all 7 genuine orders. Order 2 (0.88) beats all 7. Order 4 (0.72) beats 6 of them, all except order 3. Order 7 (0.47) beats 4. Order 11 (0.15) beats only order 12. That is 7 + 7 + 6 + 4 + 1 = 25 wins out of 35, and 25 / 35 \approx 0.71, exactly the shaded area. AUC measures how well the model ranks, and it does not depend on any threshold.

Why ROC-AUC can mislead on rare events, and the precision–recall curve

Here is the catch. Imagine a real test set with 100 frauds and 99,900 genuine orders. Flagging 1,000 genuine orders by mistake is a disaster for the reviewers: they would see ten false alarms for every real fraud. But on the ROC curve's horizontal axis, 1,000 out of 99,900 is a false positive rate of about 0.01, a tiny step to the right. The ROC curve barely notices. Because its horizontal axis divides by the enormous number of genuine orders, ROC-AUC can look excellent while precision is terrible.

The right panel fixes this. The precision–recall curve plots precision against recall as the threshold moves. Precision divides by the number of flagged orders, so every false alarm counts heavily, just as it does for the reviewers. The dashed line shows the precision you would get by flagging everything, which is simply the fraction of orders that are fraud, here 5/12. On a real fraud dataset that line sits at 0.005, and the gap between the curve and that line is what the model is really worth. For any problem where the positive class is rare, which includes fraud, disease, defects, churn and most interesting problems, look at the precision–recall curve, not just ROC-AUC.

4. Choosing the threshold from money

Now the decision that actually matters: which threshold does Cartwise use? The answer comes from the confusion matrix and two prices, the "price each kind of mistake" step from 12.1.3.

• A false positive blocks a genuine customer. Cartwise loses the margin on that sale, pays for a reviewer's time, and risks losing the customer. Call it ₹300.
• A false negative ships a fraudulent order. Cartwise loses the goods and pays a dispute fee. Call it ₹4,000.

Missing a fraud costs more than thirteen times as much as a false alarm. A threshold of 0.5 quietly assumes the two mistakes cost the same, so it is almost certainly wrong here. The way to find the right one is to try every threshold and add up the cost:

import numpy as np

scores = np.array([0.95, 0.88, 0.81, 0.72, 0.64, 0.55, 0.47, 0.38, 0.30, 0.22, 0.15, 0.08])
fraud  = np.array([1,    1,    0,    1,    0,    0,    1,    0,    0,    0,    1,    0])
COST_FP, COST_FN = 300, 4_000                                  # (1) rupees per mistake

def total_cost(t):
    flagged = scores >= t                                      # (2)
    fp = np.sum(flagged & (fraud == 0))                        # (3) flagged but genuine
    fn = np.sum(~flagged & (fraud == 1))                       # (4) passed but fraud
    return fp * COST_FP + fn * COST_FN

for t in [0.9, 0.7, 0.5, 0.4, 0.1]:
    print(t, total_cost(t))
# → 0.9 16000
# → 0.7 8300
# → 0.5 8900
# → 0.4 4900
# → 0.1 1800                                                   # (5)

Line (1) writes the business costs down as numbers, which is the most important line in the snippet: until someone puts prices on the two mistakes, threshold choice is guesswork. Line (2) turns scores into decisions at threshold t, giving an array of True and False. Line (3) counts flagged orders that were genuine, using & to mean "and" element by element. Line (4) counts fraud that was passed, where ~ means "not". Line (5) shows the result: the cheapest threshold on this data is 0.1, costing ₹1,800, against ₹8,900 at the default 0.5.

So the right threshold is far lower than the default, because a missed fraud is so expensive. But notice what happens at 0.1: 11 of the 12 orders are flagged. That is fine for twelve orders, but on 40,000 orders a day it might mean 20,000 reviews, which the team cannot do. In practice the cost calculation includes a third constraint, the reviewer capacity, and the final threshold is the cheapest one the team can actually staff. This is a business conversation that the confusion matrix makes possible, and it is the conversation an AI engineer should be leading.

There is also a neat shortcut when the probabilities are trustworthy (section 5). Flag an order when the expected cost of passing it is bigger than the expected cost of flagging it: p \times 4000 > (1 - p) \times 300. Solving for p gives p > 300 / (300 + 4000) \approx 0.07. That matches the sweep: the best threshold is low, around 0.07 to 0.1. This is the expected-value reasoning of 12.2.5 turned into a formula.

5. Calibration: can the probabilities be believed?

The formula above only works if a score of 0.07 really means a 7 percent chance of fraud. A model whose scores mean what they say is calibrated: among all orders it scores around 0.7, about 70 percent should turn out to be fraud. The weather forecast is the classic example: if a forecaster says "70 percent chance of rain" on a hundred different days, it should rain on about seventy of them.

A reliability diagram. A calibrated model's points sit on the diagonal. The red model ranks orders well enough, but its high scores are overconfident: orders it calls 90 percent fraud are fraud only about 60 percent of the time.

The chart is called a reliability diagram, and the amber box shows how to build one. Its message is simple: if the curve sits on the diagonal, you can plug the scores into cost formulas; if it bends away, you cannot, even if the AUC is excellent. Ranking well and being calibrated are different properties. AUC only checks the order of the scores. Calibration checks the values.

Some models come out reasonably calibrated: logistic regression usually does, because it is trained directly with cross-entropy on probabilities. Others do not. Deep neural networks are often overconfident, and gradient-boosted trees and models trained on rebalanced data (Chapter 12.4.3) often need correcting. The correction is a small extra step after training, fitted on held-out data:

• Platt scaling, after the computer scientist John Platt, fits a tiny logistic regression that takes the model's score as its only input and outputs a corrected probability.
• Isotonic regression fits a staircase that can bend any way, as long as it never goes down, so it can fix more unusual shapes but needs more data.
• Temperature scaling, used for neural networks, divides the logits by one learned temperature before the softmax, exactly the temperature of 12.2.6. One number, learned on held-out data, often fixes most of a network's overconfidence.

Two numbers summarise calibration. The Brier score is the mean squared difference between the predicted probability and the 0-or-1 truth. For our twelve orders it is about 0.23 (0 is perfect, and always predicting 0.5 would score 0.25), which says this toy model is barely better calibrated than a coin. The expected calibration error is the average gap between the reliability diagram and the diagonal, weighted by how many predictions fall in each bucket.

6. Measuring regression models

For models that predict a number, like delivery days, the confusion matrix does not apply. The errors are distances, and three measures cover nearly everything:

• Mean absolute error (MAE): the average size of the error, from 12.1.3. "On average, we are off by 0.6 days." Every day of error counts the same.
• Mean squared error (MSE): the average squared error, which is what linear regression minimises. Squaring makes a 2-day error four times as bad as a 1-day error. Its units are "days squared", which nobody can picture.
• Root mean squared error (RMSE): the square root of the MSE. Back in days, but still punishing big errors more.

Choose by asking how the business feels about large errors. If a 4-day error is just twice as bad as a 2-day error, use MAE. If a 4-day error means the customer cancels and never returns, while a 2-day error means a mild grumble, use RMSE, because it cares more about the occasional disaster. And always compare against the baseline: an MAE of 0.6 days means nothing until you know the lookup-table baseline scored 0.625.

7. The same numbers from a library

All of this is built into scikit-learn, and in real work you would use it:

from sklearn.metrics import (confusion_matrix, precision_score, recall_score,
                             f1_score, roc_auc_score, average_precision_score,
                             brier_score_loss)

pred = (scores >= 0.5).astype(int)
print(confusion_matrix(fraud, pred))        # (1) → [[4 3]
                                            #        [2 3]]
print(precision_score(fraud, pred),         # → 0.5
      recall_score(fraud, pred),            # → 0.6
      round(f1_score(fraud, pred), 3))      # → 0.545
print(round(roc_auc_score(fraud, scores), 3))   # (2) → 0.714
print(round(average_precision_score(fraud, scores), 3))  # (3) → 0.755, area under the PR curve
print(round(brier_score_loss(fraud, scores), 3))         # → 0.23

Line (1) prints the confusion matrix, with one trap: scikit-learn puts the genuine class first, so the top row is [TN, FP] = [4, 3] and the bottom row is [FN, TP] = [2, 3]. That is the reverse of how the figure above is drawn, and reading the two the same way is a common mistake. Line (2) gives the same 0.714 we got by counting pairs, and notice that it takes the raw scores, not the thresholded pred, because AUC covers every threshold at once. Line (3) computes the area under the precision–recall curve, which is the number to report for rare-event problems.

What the interviewer will push on

"Your fraud model has 99 percent accuracy. Is it good?" They are checking whether you ask about the base rate before saying anything else. The tell of a good answer is that you immediately ask what fraction of cases are fraud, and then talk about precision and recall at a stated threshold. The common wrong answer is to discuss the accuracy number at all.

"Precision or recall: which matters more here?" They are checking that you connect metrics to costs. A strong answer names who pays for each mistake (customers for false positives, the business for false negatives), puts rough prices on them, and says the threshold should come from those prices. The wrong answer is a fixed rule like "recall always matters more for fraud".

"Why not use ROC-AUC?" They want to hear that ROC-AUC is threshold-free and measures ranking, but on heavily imbalanced data it hides poor precision because the false positive rate divides by a huge number of negatives. The tell is that you volunteer the precision–recall curve and the base-rate line.

"The model says 0.8. What does that mean?" They are checking that you know about calibration. A good answer separates ranking quality from probability quality, mentions a reliability diagram, and names a fix such as Platt scaling or temperature scaling fitted on held-out data.

"How would you pick the threshold?" The answer they want is a cost sweep, or the formula p > C_{FP} / (C_{FP} + C_{FN}), with the operational constraint of review capacity added on top. A memorised "0.5" is the wrong answer.

Volunteer this: report every metric with the number of positive examples it was measured on. A recall of 0.9 measured on 50 frauds has a standard error of about 0.04, from 12.2.5, so two models that differ by two points on that test set have not been shown to differ at all.

Recall

• Accuracy is misleading when one class is rare: always-say-genuine scores 99.5 percent at a 0.5 percent fraud rate.
• The confusion matrix splits outcomes into true positive, false negative, false positive and true negative. The second word is what the model did; the first is whether it was right.
• Precision = TP / (TP + FP), the reviewers' number. Recall = TP / (TP + FN), the finance team's number. The threshold trades one for the other. The F1 score is their harmonic mean.
• The ROC curve and its AUC measure ranking across all thresholds; AUC is the chance a random positive outscores a random negative. On rare events, use the precision–recall curve.
• Choose the threshold from money: sweep thresholds and total the cost, or flag when p > C_{FP}/(C_{FP}+C_{FN}), then respect review capacity. That only works if the model is calibrated; check with a reliability diagram, fix with Platt, isotonic or temperature scaling. For regression, choose between MAE and RMSE by how bad large errors are.

Self-test: At a threshold of 0.7 on the twelve orders, compute precision and recall from the table. If a false positive cost ₹1,000 and a false negative ₹1,000, what probability threshold does the formula give, and why? Why does roc_auc_score take scores rather than 0-or-1 predictions?

Next: 12.3.3 Overfitting and Generalisation — these measurements are only honest if they come from data the model never learned from; the next page is about why models that look perfect in training fail on new data, and how to keep your measurements truthful.