Evaluation & Calculate Top-N Accuracy: Top 1 and Top 5

Evaluating and Calculating Top-N Accuracy: Top 1 and Top 5

What is Top-N Accuracy?

Top-N accuracy is a metric used in information retrieval and machine learning to evaluate the performance of a model, especially in ranking or classification tasks. It measures the percentage of times the correct answer is found within the top N predicted results. For instance:

• Top 1 accuracy: The model correctly predicts the answer in the first position.
• Top 5 accuracy: The model correctly predicts the answer within the top five positions.

How to Calculate Top-N Accuracy

Let’s illustrate the calculation with a simple example:

In this example:

• Top 1 accuracy: 1/3 (33.3%): The model correctly predicts the answer in the first position only once (Dog).
• Top 5 accuracy: 2/3 (66.7%): The model correctly predicts the answer within the top five positions twice (Dog, Car).

Code Example: Python

Here’s a Python code snippet to calculate Top 1 and Top 5 accuracy:

import numpy as np

def calculate_top_n_accuracy(y_true, y_pred, n=1):
  """
  Calculates Top-N accuracy.

  Args:
      y_true: True labels.
      y_pred: Predicted labels.
      n: Top N to consider.

  Returns:
      Top-N accuracy.
  """
  top_n_indices = np.argsort(y_pred, axis=1)[:, -n:]
  correct_predictions = np.sum(np.take_along_axis(y_true, top_n_indices, axis=1) == 1, axis=1)
  return np.mean(correct_predictions)

# Example usage:
y_true = np.array([1, 0, 1])
y_pred = np.array([[0.1, 0.9], [0.6, 0.4], [0.2, 0.8]])

top_1_accuracy = calculate_top_n_accuracy(y_true, y_pred, n=1)
top_5_accuracy = calculate_top_n_accuracy(y_true, y_pred, n=5)

print(f"Top 1 Accuracy: {top_1_accuracy}")
print(f"Top 5 Accuracy: {top_5_accuracy}")

When to Use Top-N Accuracy

Top-N accuracy is particularly useful in scenarios where:

• Ranking is crucial: Like in search engines or recommendation systems, where getting the most relevant items at the top is critical.
• Multiple correct answers are possible: For example, in image tagging, multiple tags could be correct for a single image.
• Measuring overall performance: It provides a comprehensive view of the model’s ability to predict relevant results within a specified range.

Conclusion

Top-N accuracy offers a valuable evaluation metric for models that deal with ranking and classification tasks. By considering the top N predictions, it provides a more holistic perspective on a model’s performance than just looking at the top 1 prediction.