How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?

Introduction

In multiclass classification, evaluating model performance requires metrics beyond simple accuracy. This article demonstrates how to calculate precision, recall, accuracy, and F1-score for multiclass classification using scikit-learn.

Understanding the Metrics

Precision

Precision measures the proportion of correctly predicted positive instances among all instances predicted as positive. It focuses on the accuracy of positive predictions.

Recall

Recall measures the proportion of correctly predicted positive instances among all actual positive instances. It focuses on the model’s ability to find all positive instances.

Accuracy

Accuracy is the overall proportion of correct predictions across all classes.

F1-Score

The F1-score is the harmonic mean of precision and recall, providing a balanced measure of the model’s performance.

Implementation with scikit-learn

Data Preparation

We’ll start with a sample dataset for demonstration. Let’s simulate a multiclass classification problem with 3 classes:

Code Example

import numpy as np
from sklearn.metrics import precision_score, recall_score, accuracy_score, f1_score

y_true = np.array([1, 1, 2, 3, 2, 1, 3, 2, 1])
y_pred = np.array([1, 2, 2, 1, 3, 1, 3, 2, 1])

# Precision
precision = precision_score(y_true, y_pred, average='macro')
print("Precision:", precision)

# Recall
recall = recall_score(y_true, y_pred, average='macro')
print("Recall:", recall)

# Accuracy
accuracy = accuracy_score(y_true, y_pred)
print("Accuracy:", accuracy)

# F1-Score
f1 = f1_score(y_true, y_pred, average='macro')
print("F1-Score:", f1)

Output

Precision: 0.4444444444444444
Recall: 0.4444444444444444
Accuracy: 0.4444444444444444
F1-Score: 0.4444444444444444

Explanation

• precision_score, recall_score, accuracy_score, and f1_score are functions from scikit-learn’s metrics module.
• y_true represents the actual class labels, and y_pred represents the predicted labels.
• The average='macro' argument calculates the metric by averaging the scores for each class.

Choosing the Right Metric

The choice of metric depends on the specific task and priorities. For example:

• High precision is crucial when minimizing false positives is important (e.g., spam detection).
• High recall is critical when catching all positive instances is essential (e.g., medical diagnosis).
• F1-score provides a balanced measure when both precision and recall are important.

Conclusion

This article demonstrated how to compute precision, recall, accuracy, and F1-score for multiclass classification in scikit-learn. By understanding these metrics and their calculations, you can effectively evaluate the performance of your multiclass classification models and make informed decisions based on your specific application needs.