Precision, Recall and F1
Precision asks how many of your positive predictions were right; recall asks how many of the real positives you caught. They trade off against each other, F1 is their harmonic mean, and accuracy lies to you the moment the classes are imbalanced.
Transcript
Here's the follow-up interviewers hold in reserve. Your model has 99% accuracy. Ship it? The strong answer doesn't agree to anything until it has seen the class balance and the confusion matrix, because on rare classes that one accuracy number can hide a model that catches nothing at all.
Take one worked example. A thousand transactions, ten of them fraud, and the model flags thirty, of which eight are real fraud. Those eight are true positives. The other twenty-two flags are false positives, the false alarms, two frauds slipped through as false negatives, and the remaining 968 transactions are true negatives, correctly left alone.
Precision reads down the column of things you flagged: of everything you raised your hand for, what fraction was correct? Eight out of thirty, which is 0.267. Most flags are noise.
Recall reads across the row of things that were really fraud. Of everything that was actually positive, what fraction did you catch? Eight out of ten, so 0.8. Notice what's missing from both. Precision ignores the true negatives entirely, and recall ignores the false positives as well as the true negatives.
Accuracy counts them. It's right answers over everything, so eight plus 968, which gives 976 out of a thousand, or 97.6 percent. Sounds great. The true negatives are doing almost all of the work.
Now take a model that flags nothing at all. It's right about all 990 legitimate transactions, so it scores 99 percent accuracy, with a recall of zero. Put the two accuracy figures side by side and the better model is the useless one. Rare positives? Use precision and recall.
A classifier outputs a score, and you pick a threshold, a cutoff. Here are the same ten frauds with hypothetical scores, swept at four thresholds, and as you slide the threshold, precision and recall move in opposite directions. At 0.6 the analyst reviews eight cases and misses three frauds. At 0.4 they review twenty-nine and miss one. Is that worth it? That's the business's number to supply.
So you don't optimize both. You decide which error is more expensive, then tune the threshold there. Favor precision when a false positive is costly, like a spam filter that dumps a real invoice into junk. Favor recall when a miss is costly: fraud detection, cancer screening, security alerting. There you accept extra false alarms and add a cheap second-stage review to clean them up.
F1 collapses the two into one number with the harmonic mean, which is dragged toward the smaller value. A precision of one and a recall of zero average to 0.5. Their F1 is zero. On the fraud matrix, F1 is 0.40. F2 weights recall higher, and it gives 0.57, so the same model looks better under the metric that cares about the error it's good at avoiding. Choose the metric first.
So when you hear 99% accuracy, ask for the class balance and the confusion matrix, then show the precision-recall curve, so the threshold becomes a business decision. The worked matrix is on fdeinterviews.com.
TL;DR: Precision is
TP / (TP + FP)(of what you flagged, how much was real), recall isTP / (TP + FN)(of what was real, how much you caught). They trade off, so pick the one your cost structure punishes most: precision when false alarms hurt, recall when misses hurt. F1 is their harmonic mean, and accuracy is the wrong metric on imbalanced data.
Start from the confusion matrix
Every binary classifier sorts predictions into four buckets. A true positive (TP) is a real positive you correctly flagged. A false positive (FP) is something you flagged that was actually negative, a false alarm. A false negative (FN) is a real positive you missed. A true negative (TN) is a negative you correctly left alone.
Two ratios fall out of those buckets:
precision = TP / (TP + FP)reads down the column of things you predicted positive. Of everything you raised your hand for, what fraction was correct?recall = TP / (TP + FN)reads across the row of things that were positive. Of everything that was actually positive, what fraction did you catch?
Notice precision ignores TN entirely and recall ignores both FP and TN. That is the whole reason accuracy can mislead.
One confusion matrix, worked to the end. A thousand transactions, ten of them fraud. The model flags thirty, of which eight are real fraud. So TP = 8, FP = 22, FN = 2, TN = 968. Precision is 8 / 30 = 0.267, recall is 8 / 10 = 0.8, F1 is 0.4, and accuracy is 976 / 1000 = 97.6 percent (computed). The model that flags nothing scores 99 percent accuracy with a recall of zero. Read those two accuracy figures side by side and the "better" model is the useless one, which is the whole argument of this page in two numbers.
The trade-off is real, not academic
A classifier outputs a score, and you pick a threshold above which you call it positive. Slide the threshold and the two metrics move in opposite directions. Lower the bar and you catch more real positives (recall up) but also wave through more junk (precision down). Raise the bar and you only flag the cases you are sure of (precision up) while quietly missing the borderline real ones (recall down).
So you do not optimize both. You decide which error is more expensive and tune the threshold there. Drag the threshold below and watch precision and recall pull against each other:
The same ten fraud cases against 990 legitimate ones, with hypothetical model scores, swept at four thresholds (computed):
| Threshold | Flagged | Fraud caught | False alarms | Precision | Recall |
|---|---|---|---|---|---|
| 0.8 | 3 | 3 | 0 | 1.00 | 0.3 |
| 0.6 | 8 | 7 | 1 | 0.88 | 0.7 |
| 0.4 | 29 | 9 | 20 | 0.31 | 0.9 |
| 0.2 | 218 | 10 | 208 | 0.05 | 1.0 |
At 0.6 the analyst reviews eight cases a day and misses three frauds; at 0.4 they review twenty-nine and miss one. Whether the twenty-one extra reviews are worth two more caught frauds is not a modeling question. It is the cost of an analyst-hour against the cost of a missed case, and it is the business's number to supply.
- Favor precision when a false positive is costly: a spam filter that dumps a real invoice into junk, or a content filter that blocks a paying customer. You would rather let a little spam through than lose a legitimate email.
- Favor recall when a false negative is costly: fraud detection, cancer screening, security alerting. Missing a real case is the expensive failure, so you accept extra false alarms and add a cheap second-stage review to clean them up.
F1 and why the harmonic mean
F1 collapses precision (P) and recall (R) into one number with F1 = 2*P*R / (P + R). It uses the harmonic mean, not the plain average, on purpose: the harmonic mean is dragged toward the smaller value. A classifier with precision 1.0 and recall 0.0 has an arithmetic mean of 0.5 but an F1 of 0. That punishes the lazy "predict everything positive" or "predict nothing positive" tricks that game one metric while ignoring the other.
When the two errors are not equally costly, use F-beta. F2 weights recall higher (good for fraud), F0.5 weights precision higher (good for spam). On the fraud matrix above, F1 is 0.40 and F2 is 0.57 (computed): the same model looks better under the metric that cares about the error it is good at avoiding, which is the point of choosing the metric before looking at the score.
Why accuracy misleads on imbalanced data
Suppose 1 in 1000 transactions is fraud. A model that predicts "not fraud" for everything scores 99.9% accuracy and catches zero fraud. Accuracy looks great because TN dominates the count, and accuracy rewards TN. Precision and recall do not, which is exactly why you reach for them whenever the positive class is rare.
Why interviewers probe this
This is the fastest way to tell whether someone has shipped a model or only read about one. The screen is not the definitions, it is the judgment: given a problem, can you name which error is worse and pick the metric that reflects it? The follow-up they hold in reserve is "your model has 99% accuracy, ship it?" The strong answer asks for the class balance and the confusion matrix before agreeing to anything, then proposes a precision-recall curve so the threshold becomes a business decision rather than a default of 0.5.
A second probe is the threshold itself. Many candidates report a single F1 and stop. Better: show that you would expose the precision-recall trade-off as a curve and let the cost of each error pick the operating point.
Common misconceptions
- "High accuracy means a good model." On a 99-to-1 class split, predicting the majority class always gives 99% accuracy and zero useful behavior. Report precision and recall, or the area under the precision-recall curve.
- "F1 is always the right summary." F1 weights precision and recall equally. If a miss costs ten times a false alarm, F1 hides that; use F-beta or report both numbers.
- "Precision and recall are independent dials." They move together through the threshold. You cannot push both up by tuning the threshold alone; that requires a better model or better features.
- "The default 0.5 threshold is correct." It is an arbitrary starting point. The right threshold comes from the relative cost of FP and FN, not the library default.
Key takeaways
- Precision = of what you flagged, how much was real; recall = of what was real, how much you caught.
- Pick the metric your cost structure punishes: precision when false alarms hurt, recall when misses hurt.
- F1 is the harmonic mean of the two, so it collapses to zero if either is zero; use F-beta when the errors are not equally costly.
- Accuracy is misleading on imbalanced data because it rewards the easy majority class; always check the confusion matrix.