True Positives, False Positives, True Negatives, False Negatives

While accuracy gives us a general sense of how often our model is right, it doesn't tell the whole story. Imagine an email spam filter. If only 1% of emails are actually spam, a model that always predicts "not spam" would be 99% accurate! That sounds great, but it completely fails at its primary task: catching spam. To get a better understanding, we need to break down the model's predictions into more specific categories.

For classification tasks, especially binary classification (where there are two possible outcomes, like "spam" vs. "not spam" or "disease" vs. "healthy"), we analyze predictions by comparing them to the actual, true values. We typically designate one class as "Positive" and the other as "Negative". Which class is positive is often determined by the focus of the problem (e.g., detecting spam, identifying a disease).

Based on this comparison, every prediction falls into one of four categories:

True Positives (TP)

These are the cases where the model correctly predicts the positive class.

• Actual: Positive
• Predicted: Positive
• Example: An email that is spam is correctly identified as spam by the filter.

False Positives (FP)

These are the cases where the model incorrectly predicts the positive class when the actual class is negative. This is sometimes called a "Type I Error".

• Actual: Negative
• Predicted: Positive
• Example: An important email that is not spam is incorrectly identified as spam and perhaps sent to the junk folder.

True Negatives (TN)

These are the cases where the model correctly predicts the negative class.

• Actual: Negative
• Predicted: Negative
• Example: An email that is not spam is correctly identified as not spam.

False Negatives (FN)

These are the cases where the model incorrectly predicts the negative class when the actual class is positive. This is sometimes called a "Type II Error".

• Actual: Positive
• Predicted: Negative
• Example: An email that is spam is incorrectly identified as not spam and lands in the inbox.

Understanding the Outcomes

We can summarize these four possibilities in a table format, which forms the basis of the Confusion Matrix we will discuss next:

	Predicted: Positive	Predicted: Negative
Actual: Positive	True Positive (TP)	False Negative (FN)
Actual: Negative	False Positive (FP)	True Negative (TN)

Think about the total number of predictions your model makes. Every single prediction fits into one of these four boxes (TP, FP, TN, FN). Therefore, the total number of instances is:

$Total = TP + FP + TN + FN$

Accuracy, which we discussed earlier, can also be defined using these terms:

$Accuracy = \frac{TP + TN}{TP + FP + TN + FN}$

This shows that accuracy simply measures the proportion of correct predictions (both positive and negative) out of all predictions made.

Breaking down predictions into these four categories is fundamental because different types of errors often have very different consequences. In our spam example:

• A False Positive (FP) means a legitimate email might be missed. This can be annoying or even problematic.
• A False Negative (FN) means a spam email gets through. This is also annoying.

In other scenarios, the cost imbalance can be much more severe. Consider a medical diagnosis model predicting a serious disease (Positive class = Disease):

• A False Positive (FP) means a healthy patient is told they might have the disease, leading to anxiety and further testing (which might be costly or invasive).
• A False Negative (FN) means a patient with the disease is told they are healthy, potentially delaying life-saving treatment.

Clearly, depending on the application, we might want to minimize one type of error more than the other. Understanding TP, FP, TN, and FN allows us to move past simple accuracy and choose metrics like Precision and Recall (discussed next) that better reflect the specific goals and constraints of our classification problem.