Precision, Recall and F1

TL;DR: Precision is TP / (TP + FP) (of what you flagged, how much was real), recall is TP / (TP + FN) (of what was real, how much you caught). They trade off, so pick the one your cost structure punishes most: precision when false alarms hurt, recall when misses hurt. F1 is their harmonic mean, and accuracy is the wrong metric on imbalanced data.

Start from the confusion matrix

Every binary classifier sorts predictions into four buckets. A true positive (TP) is a real positive you correctly flagged. A false positive (FP) is something you flagged that was actually negative, a false alarm. A false negative (FN) is a real positive you missed. A true negative (TN) is a negative you correctly left alone.

Two ratios fall out of those buckets:

• precision = TP / (TP + FP) reads down the column of things you predicted positive. Of everything you raised your hand for, what fraction was correct?
• recall = TP / (TP + FN) reads across the row of things that were positive. Of everything that was actually positive, what fraction did you catch?

Notice precision ignores TN entirely and recall ignores both FP and TN. That is the whole reason accuracy can mislead.

The trade-off is real, not academic

A classifier outputs a score, and you pick a threshold above which you call it positive. Slide the threshold and the two metrics move in opposite directions. Lower the bar and you catch more real positives (recall up) but also wave through more junk (precision down). Raise the bar and you only flag the cases you are sure of (precision up) while quietly missing the borderline real ones (recall down).

So you do not optimize both. You decide which error is more expensive and tune the threshold there. Drag the threshold below and watch precision and recall pull against each other:

• Favor precision when a false positive is costly: a spam filter that dumps a real invoice into junk, or a content filter that blocks a paying customer. You would rather let a little spam through than lose a legitimate email.
• Favor recall when a false negative is costly: fraud detection, cancer screening, security alerting. Missing a real case is the expensive failure, so you accept extra false alarms and add a cheap second-stage review to clean them up.

F1 and why the harmonic mean

F1 collapses precision (P) and recall (R) into one number with F1 = 2*P*R / (P + R). It uses the harmonic mean, not the plain average, on purpose: the harmonic mean is dragged toward the smaller value. A classifier with precision 1.0 and recall 0.0 has an arithmetic mean of 0.5 but an F1 of 0. That punishes the lazy "predict everything positive" or "predict nothing positive" tricks that game one metric while ignoring the other.

When the two errors are not equally costly, use F-beta. F2 weights recall higher (good for fraud), F0.5 weights precision higher (good for spam).

Why accuracy misleads on imbalanced data

Suppose 1 in 1000 transactions is fraud. A model that predicts "not fraud" for everything scores 99.9% accuracy and catches zero fraud. Accuracy looks great because TN dominates the count, and accuracy rewards TN. Precision and recall do not, which is exactly why you reach for them whenever the positive class is rare.

Why interviewers probe this

This is the fastest way to tell whether someone has shipped a model or only read about one. The screen is not the definitions, it is the judgment: given a problem, can you name which error is worse and pick the metric that reflects it? The follow-up they hold in reserve is "your model has 99% accuracy, ship it?" The strong answer asks for the class balance and the confusion matrix before agreeing to anything, then proposes a precision-recall curve so the threshold becomes a business decision rather than a default of 0.5.

A second probe is the threshold itself. Many candidates report a single F1 and stop. Better: show that you would expose the precision-recall trade-off as a curve and let the cost of each error pick the operating point.

Common misconceptions

• "High accuracy means a good model." On a 99-to-1 class split, predicting the majority class always gives 99% accuracy and zero useful behavior. Report precision and recall, or the area under the precision-recall curve.
• "F1 is always the right summary." F1 weights precision and recall equally. If a miss costs ten times a false alarm, F1 hides that; use F-beta or report both numbers.
• "Precision and recall are independent dials." They move together through the threshold. You cannot push both up by tuning the threshold alone; that requires a better model or better features.
• "The default 0.5 threshold is correct." It is an arbitrary starting point. The right threshold comes from the relative cost of FP and FN, not the library default.

Key takeaways

• Precision = of what you flagged, how much was real; recall = of what was real, how much you caught.
• Pick the metric your cost structure punishes: precision when false alarms hurt, recall when misses hurt.
• F1 is the harmonic mean of the two, so it collapses to zero if either is zero; use F-beta when the errors are not equally costly.
• Accuracy is misleading on imbalanced data because it rewards the easy majority class; always check the confusion matrix.