Code › codeit-ai-sprint
Confusion Matrices and Classification Metrics
How accuracy, recall, precision, and F1 expose different kinds of classification errors
In my previous post on logistic regression and decision boundaries, I looked at how a model can produce a probability such as an 80% chance that a product is defective. A threshold then turns that probability into a final class: defective or normal.
I paid particular attention to this part of the course because I had already encountered these metrics while building Tail Villain. I needed a way to check the quality of a multi-turn LLM interviewer and coach, so I built an interview evaluation system. Precision, recall, and F1 also appeared in the admin evaluation page. This class gave me a chance to work through what those numbers actually count.
Why Accuracy Is Not Enough
Once a model has produced its final classifications, how should those results be evaluated? Accuracy is the most immediate answer. It is simply the proportion of all predictions that were correct.
Accuracy becomes less informative when one class greatly outnumbers the other. Credit card fraud detection, spam filtering, medical screening, and defect inspection are common examples. If only 1% of products are defective, a model that labels every product as normal still reaches 99% accuracy while finding none of the defects.
Dividing predictions into correct and incorrect results hides the kind of mistake the model made. Comparing the actual and predicted classes produces four cases, which reveal the errors hidden inside the accuracy score.
The Four Outcomes in a Confusion Matrix
A confusion matrix crosses the actual class with the predicted class.
| Actual class | Predicted Positive | Predicted Negative |
|---|---|---|
| Positive | True Positive, TP | False Negative, FN |
| Negative | False Positive, FP | True Negative, TN |
Positive does not mean that the outcome is good. It identifies the event the model is trying to find. Positive can mean that a patient has a disease, a product is defective, or a message is spam.
True and False indicate whether the prediction matches the actual class. Positive and Negative indicate which class the model predicted. The abbreviations TP, FN, FP, and TN are used in the formulas that follow.
True Positive means the model correctly predicted a positive example as positive, while True Negative means it correctly predicted a negative example as negative. Both are correct predictions.
False Positive means the model predicted positive for an example that was actually negative. Marking a normal product as defective falls into this category. False Negative is the opposite: the model misses an actual positive example by predicting negative, such as allowing a defective product to pass inspection.
I initially found the four names confusing when I tried to remember each one separately. They became easier to distinguish once I read Positive or Negative as the model’s decision and True or False as whether that decision was correct.
Connecting the Metrics with One Example
Suppose 10 out of 1,000 products are actually defective. The model predicts that 15 products are defective, and 8 of those predictions are correct.
| Actual class | Predicted defective | Predicted normal | Total |
|---|---|---|---|
| Defective | TP = 8 | FN = 2 | 10 |
| Normal | FP = 7 | TN = 983 | 990 |
| Total | 15 | 985 | 1,000 |
The model correctly classified 8 defective products and 983 normal products, so its accuracy is 99.1%.
That number makes the model look almost perfect. The confusion matrix tells a less comfortable story: it missed two defective products and incorrectly rejected seven normal ones. Recall and precision examine those two errors from different directions.
Recall: How Many Actual Defects Did the Model Find?
Recall is the proportion of actual positive examples that the model identified as positive.
The model found 8 of the 10 defective products, so its recall is 80%.
Recall matters when a False Negative is expensive. Missing a disease during screening or allowing a defective product to ship can cause more harm than raising an extra warning. In those cases, the evaluation needs to show how many actual positive examples the model failed to find.
Precision: How Reliable Is a Positive Prediction?
An 80% recall score does not tell us how many normal products the model rejected while finding those eight defects. Precision answers that question by measuring the proportion of positive predictions that were actually positive.
The model predicted 15 defective products, but only 8 were truly defective. Its precision is therefore about 53.3%.
Precision matters when a False Positive is expensive. Sending legitimate email to a spam folder or discarding a normal product can be costly, so a positive prediction should not be made too freely.
Recall and precision share TP in the numerator, but their denominators are different. Recall starts from all actual positive examples. Precision starts from everything the model predicted as positive. Remembering which set forms the denominator is more useful than memorizing the two formulas in isolation.
The Threshold Changes Recall and Precision
Logistic regression outputs a probability between 0 and 1, then compares it with a threshold to select the final class. Lowering the threshold classifies more examples as positive. This can reduce False Negatives and raise recall, but it can also create more False Positives and lower precision.
Raising the threshold has the opposite effect. The model predicts positive only when it is more confident, which can reduce False Positives and improve precision. It can also miss more actual positive examples, increasing False Negatives and lowering recall.
lower the threshold
→ more positive predictions
→ fewer FN, potentially more FP
→ recall may rise, precision may fall
raise the threshold
→ fewer positive predictions
→ fewer FP, potentially more FN
→ precision may rise, recall may fallRecall and precision do not always move in perfectly opposite proportions. Still, this trade-off appears frequently when the threshold changes on the same model. The threshold cannot be chosen from model output alone. It depends on whether a False Positive or a False Negative creates the greater real-world cost.
Using F1 to Consider Both Metrics
The F1 score is the harmonic mean of precision and recall.
In the product example, precision is about 0.533 and recall is 0.8, producing an F1 score of about 0.64.
Because F1 uses a harmonic mean rather than an arithmetic mean, it drops sharply when either precision or recall is low. A high F1 score requires both metrics to remain reasonably high.
F1 is not a replacement for the underlying metrics. Two models can have the same F1 score with different combinations of precision and recall, and True Negatives do not appear directly in the formula. Even when F1 is used for comparison, precision and recall still need to be checked to understand which errors the model makes.
This review made the numbers in Tail Villain’s admin evaluation page more concrete. Adding metrics does not complete an evaluation system by itself, but they can provide a consistent comparison before and after changing a model or prompt. The important part is knowing which errors each number counts instead of treating the highest score as the answer.