A retrieval system's top 5 results by similarity alone turn out to be five near-duplicate passages that all restate the same single fact, because the corpus happens to contain many redundant copies of that fact and none of the closest embeddings differ much from each other. A technique called Maximal Marginal Relevance (MMR) re-ranks the candidate pool to fix exactly this problem. How does it work?
- It picks each next result by rewarding closeness to the query but penalizing closeness to results already picked, so that once a fact has been represented once, near-duplicate passages restating it score lower and passages covering different information get a chance to be selected instead
- It removes any passage whose embedding is closer to another passage's embedding than a fixed distance threshold, deleting near-duplicates from the corpus entirely before any query is ever run
- It retrains the embedding model so that semantically similar passages are pushed further apart in vector space, permanently reducing how many near-duplicate passages the corpus can contain
- It runs the query once against each half of the corpus separately and interleaves the two result lists so that whichever half a passage was drawn from, at least some diversity across halves is guaranteed
Why A? And why not the others?
Correct answer: A. It picks each next result by rewarding closeness to the query but penalizing closeness to results already picked, so that once a fact has been represented once, near-duplicate passages restating it score lower and passages covering different information get a chance to be selected instead
MMR selects results one at a time, scoring each remaining candidate as a trade-off between how relevant it is to the query and how similar it is to results already chosen; the first pick is whichever candidate is most relevant, but every later pick is penalized for resembling an already-selected result, so once one passage restating a fact has been chosen, near-identical restatements of that same fact score poorly against the diversity penalty while passages covering different information score relatively better and rise into the result set instead. Deleting corpus passages within a fixed distance threshold ahead of time is not how MMR works; MMR operates on the ranked candidate list at query time rather than editing the underlying corpus, and a fixed global threshold would not adapt to which passages a particular query happened to retrieve. Retraining the embedding model to push similar passages apart is also not what MMR does; MMR leaves the embeddings and the corpus untouched and only changes which of the already-computed nearest neighbors get selected into the final result list. Splitting the corpus in half and interleaving results is an arbitrary partitioning scheme that has no connection to MMR's actual relevance-versus-redundancy trade-off and would not reliably avoid near-duplicate results at all.
Source: Carbonell & Goldstein, 'The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries' (1998), https://aclanthology.org/X98-1025.pdf