How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?

Introduction

In multiclass classification, we deal with problems where there are more than two classes. Metrics like precision, recall, accuracy, and F1-score are crucial for evaluating the performance of our models. This article will guide you through computing these metrics for multiclass classification using Scikit-learn.

Understanding the Metrics

Precision

Precision measures the proportion of correctly predicted positive instances out of all instances predicted as positive. It addresses the question: “Of all the instances predicted to belong to a specific class, how many are actually from that class?”

Recall

Recall measures the proportion of correctly predicted positive instances out of all actual positive instances. It answers the question: “Of all the instances actually belonging to a specific class, how many were correctly predicted?”

Accuracy

Accuracy is the overall proportion of correctly classified instances across all classes. It reflects the overall performance of the model.

F1-score

F1-score is the harmonic mean of precision and recall. It provides a balanced measure considering both precision and recall.

Computing Metrics with Scikit-learn

Scikit-learn offers convenient tools to calculate these metrics. Let’s walk through an example:

Example: Iris Dataset

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# Load Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train a Logistic Regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predict on the test set
y_pred = model.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)

# Calculate precision
precision = precision_score(y_test, y_pred, average='macro')

# Calculate recall
recall = recall_score(y_test, y_pred, average='macro')

# Calculate F1-score
f1 = f1_score(y_test, y_pred, average='macro')

# Print results
print(f"Accuracy: {accuracy}")
print(f"Precision: {precision}")
print(f"Recall: {recall}")
print(f"F1-score: {f1}")

Output:

Accuracy: 1.0
Precision: 1.0
Recall: 1.0
F1-score: 1.0

Explanation:

• We load the Iris dataset and split it into training and testing sets.
• A Logistic Regression model is trained on the training data.
• Predictions are made on the test set.
• The accuracy_score, precision_score, recall_score, and f1_score functions are used to compute the respective metrics. The average='macro' parameter computes the metric for each class and averages them.

Choosing the Right Metric

The choice of the most suitable metric depends on the specific problem and priorities:

• Accuracy is a good overall measure of model performance, but may be misleading when dealing with imbalanced datasets.
• Precision is crucial when minimizing false positives is important, e.g., in medical diagnosis.
• Recall is more important when minimizing false negatives is critical, e.g., in fraud detection.
• F1-score provides a balanced measure considering both precision and recall, making it useful in many scenarios.

Conclusion

By understanding these metrics and utilizing Scikit-learn’s powerful functions, you can effectively evaluate the performance of multiclass classification models and make informed decisions about your machine learning solutions.