From Surf Wiki (app.surf) — the open knowledge base

Mean reciprocal rank

Search quality measure in information retrieval

The mean reciprocal rank is a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse of the rank of the first correct answer: 1 for first place, for second place, for third place and so on. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q:

: \text{MRR} = \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{\text{rank}_i}. !

where \text{rank}_i refers to the rank position of the first relevant document for the i-th query.

The reciprocal value of the mean reciprocal rank corresponds to the harmonic mean of the ranks.

Example

Suppose we have the following three queries for a system that tries to translate English words to their plurals. In each case, the system makes three guesses, with the first one being the one it thinks is most likely correct:

Query	Proposed Results	Correct response	Rank	Reciprocal rank
cat	catten, cati, cats	cats	3	1/3
torus	torii, tori, toruses	tori	2	1/2
virus	viruses, virii, viri	viruses	1	1

Given those three samples, we could calculate the mean reciprocal rank as (1/3 + 1/2 + 1) / 3 = 11/18, or approximately 0.61.

If none of the proposed results are correct, the reciprocal rank is 0. Note that only the rank of the first relevant answer is considered, and possible further relevant answers are ignored. If users are also interested in further relevant items, mean average precision is a potential alternative metric.

References

E.M. Voorhees. (1999). "Proceedings of the 8th Text Retrieval Conference".
(2002). "Evaluating web-based question answering systems".

Info: Wikipedia Source

This article was imported from Wikipedia and is available under the Creative Commons Attribution-ShareAlike 4.0 License. Content has been adapted to SurfDoc format. Original contributors can be found on the article history page.

summary-statistics information-retrieval-evaluation

Want to explore this topic further?

Ask Mako anything about Mean reciprocal rank — get instant answers, deeper analysis, and related topics.

Research with Mako

Free with your Surf account

Content sourced from Wikipedia, available under CC BY-SA 4.0.

This content may have been generated or modified by AI. CloudSurf Software LLC is not responsible for the accuracy, completeness, or reliability of AI-generated content. Always verify important information from primary sources.

Report