Evaluation & Calculate Top-N Accuracy: Top 1 and Top 5
Introduction
Top-N accuracy is a popular evaluation metric used in information retrieval, recommender systems, and other applications where the goal is to rank items based on their relevance to a given query or user profile. It measures the percentage of times that the true relevant items are present within the top-N retrieved items.
Top-N Accuracy
Top-N accuracy, also known as “precision at N”, is calculated as follows:
<pre>
Top-N Accuracy = (Number of relevant items in top-N) / N
</pre>
For example, if we have a list of 10 recommended movies, and 3 of them are actually relevant to the user’s preferences, the top-10 accuracy would be 3/10 = 0.3.
Top 1 and Top 5 Accuracy
The most common cases are Top 1 and Top 5 accuracy.
Top 1 Accuracy
Top 1 accuracy measures the percentage of times the most relevant item is ranked at the top position. It is also known as the hit rate.
<pre>
Top 1 Accuracy = (Number of relevant items in top-1) / 1
</pre>
Top 5 Accuracy
Top 5 accuracy measures the percentage of times at least one of the top five ranked items is relevant.
<pre>
Top 5 Accuracy = (Number of relevant items in top-5) / 5
</pre>
Implementation Example
Let’s consider an example where we have a list of five recommended movies, and the actual relevant movies are indicated by “R”.
| Rank | Recommended Movie | Relevant |
|---|---|---|
| 1 | Movie A | R |
| 2 | Movie B | |
| 3 | Movie C | |
| 4 | Movie D | |
| 5 | Movie E | R |
Based on the table:
- Top 1 Accuracy = 1/1 = 1 (Movie A is relevant)
- Top 5 Accuracy = 2/5 = 0.4 (Movie A and Movie E are relevant)
Conclusion
Top-N accuracy is a valuable metric for evaluating the performance of ranking systems. Top 1 and Top 5 accuracy are commonly used to assess the ability of the system to provide the most relevant items at the top of the ranking. By analyzing these metrics, we can gain insights into the effectiveness of the ranking algorithm and identify areas for improvement.