Introduction to Confusion Matrices in Python

Introduction to Confusion Matrices

Confusion matrices are a valuable tool for evaluating the performance of a classification model. They provide a comprehensive view of how well the model is classifying instances by comparing the predicted labels with the actual labels.

from sklearn.metrics import confusion_matrix

Creating a Confusion Matrix

To create a confusion matrix using scikit-learn, we can use the confusion_matrix function from the metrics module. It takes the true labels and the predicted labels as input.

y_true = [0, 1, 0, 1, 0, 1]
y_pred = [0, 0, 1, 1, 0, 1]
conf_matrix = confusion_matrix(y_true, y_pred)
print(conf_matrix)

Understanding the Confusion Matrix Structure

A confusion matrix is a square matrix with dimensions equal to the number of classes in the dataset. The rows represent the actual classes, and the columns represent the predicted classes.

[[3 1]
 [1 2]]

Interpreting the Confusion Matrix

The diagonal elements of the confusion matrix represent the correctly classified instances, while the off-diagonal elements represent the misclassified instances.

true_positives = conf_matrix[0, 0]
false_positives = conf_matrix[0, 1]
false_negatives = conf_matrix[1, 0]
true_negatives = conf_matrix[1, 1]

Calculating Accuracy

The accuracy of a classification model can be calculated from the confusion matrix by summing the true positives and true negatives, and dividing by the total number of instances.

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(y_true, y_pred)
print(f"Accuracy: {accuracy:.2f}")

Visualizing Confusion Matrices with Matplotlib

Matplotlib is a popular Python library for data visualization. It can be used to create visual representations of confusion matrices.

import matplotlib.pyplot as plt

plt.matshow(conf_matrix, cmap=plt.cm.Blues)
plt.colorbar()
plt.show()

Improving Confusion Matrix Visualization

To enhance the visual representation of the confusion matrix, we can add labels, titles, and color maps.

import numpy as np
import matplotlib.pyplot as plt

class_names = ['Negative', 'Positive']
conf_matrix = conf_matrix.astype('float') / conf_matrix.sum(axis=1)[:, np.newaxis]

plt.imshow(conf_matrix, interpolation='nearest', cmap=plt.cm.Blues)
plt.title('Confusion Matrix')
plt.colorbar()
tick_marks = np.arange(len(class_names))
plt.xticks(tick_marks, class_names, rotation=45)
plt.yticks(tick_marks, class_names)
plt.show()

Normalized Confusion Matrix

To better interpret the confusion matrix, we can normalize it by dividing each element by the sum of its row, representing the true class proportions.

from sklearn.metrics import confusion_matrix
from sklearn.metrics import ConfusionMatrixDisplay

y_true = [0, 1, 0, 1, 0, 1]
y_pred = [0, 0, 1, 1, 0, 1]

conf_matrix = confusion_matrix(y_true, y_pred, normalize='true')
disp = ConfusionMatrixDisplay(confusion_matrix=conf_matrix)
disp.plot()
plt.show()

Precision, Recall, and F1-Score

Precision, recall, and F1-score are important metrics derived from the confusion matrix. They provide a more comprehensive evaluation of a classification model's performance.

from sklearn.metrics import precision_score, recall_score, f1_score

precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"F1-Score: {f1:.2f}")

Multi-Class Confusion Matrices

Confusion matrices can be extended to handle multi-class classification problems. The matrix dimensions increase to accommodate the additional classes.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import confusion_matrix

iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = GaussianNB()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

conf_matrix = confusion_matrix(y_test, y_pred)
print(conf_matrix)

Visualizing Multi-Class Confusion Matrices

Visualizing multi-class confusion matrices can be challenging due to the increased complexity. Scikit-learn provides a convenient function for this purpose.

from sklearn.metrics import ConfusionMatrixDisplay
import matplotlib.pyplot as plt

disp = ConfusionMatrixDisplay.from_estimator(model, X_test, y_test)
disp.ax_.set_title("Confusion Matrix")
plt.show()

Imbalanced Datasets and Confusion Matrices

Confusion matrices can be particularly useful when working with imbalanced datasets, where one class is significantly underrepresented compared to others.

from sklearn.datasets import make_blobs
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix

X, y = make_blobs(n_samples=1000, centers=2, n_features=2, cluster_std=2, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

conf_matrix = confusion_matrix(y_test, y_pred)
print(conf_matrix)

Handling Imbalanced Datasets

Imbalanced datasets can lead to biased models that favor the majority class. Techniques like oversampling, undersampling, or adjusting class weights can help mitigate this issue.

from imblearn.over_sampling import RandomOverSampler
from imblearn.under_sampling import RandomUnderSampler

over_sampler = RandomOverSampler(sampling_strategy='minority')
under_sampler = RandomUnderSampler(sampling_strategy='majority')

X_resampled, y_resampled = over_sampler.fit_resample(X_train, y_train)
X_resampled, y_resampled = under_sampler.fit_resample(X_resampled, y_resampled)

model.fit(X_resampled, y_resampled)
y_pred = model.predict(X_test)

conf_matrix = confusion_matrix(y_test, y_pred)
print(conf_matrix)

Conclusion

Confusion matrices are powerful tools for evaluating classification models. By understanding how to interpret and visualize them, you can gain valuable insights into your model's performance and identify areas for improvement.

Introduction to Confusion Matrices in Python