A RAG system sometimes retrieves passages that are only weakly related to the query, and when that happens, the generated answer still depends entirely on those weak passages because nothing in the pipeline checks how good the retrieval was before generation runs. Corrective Retrieval Augmented Generation (CRAG, Yan et al., 2024) adds a step to address this. What does it do?
- It adds a lightweight retrieval evaluator that scores the quality of the retrieved documents before generation, and when that score indicates the retrieval is poor, it triggers a corrective action such as falling back to a web search, rather than generating directly from documents already judged to be weak
- It has the language model generate special reflection tokens as part of its own output, deciding token by token whether the passages it was given were worth using at all
- It removes the retrieval step from the pipeline entirely whenever the query is judged to be a broad, corpus-wide question rather than a narrow factual one
- It re-embeds every document in the corpus using a larger embedding model whenever a low-quality retrieval is detected, then re-runs the same query against the newly re-embedded corpus
Why A? And why not the others?
Correct answer: A. It adds a lightweight retrieval evaluator that scores the quality of the retrieved documents before generation, and when that score indicates the retrieval is poor, it triggers a corrective action such as falling back to a web search, rather than generating directly from documents already judged to be weak
CRAG inserts a lightweight retrieval evaluator that assesses the overall quality of the retrieved documents for a given query and returns a confidence degree, and depending on that confidence, the pipeline can trigger a corrective knowledge-retrieval action, such as extending to a large-scale web search, rather than proceeding to generate an answer straight from documents the evaluator has already flagged as weak. This is a distinct mechanism from Self-RAG, where the model itself generates reflection tokens as part of its own output to judge, during generation, whether retrieval was needed or whether the passages it received were useful; CRAG instead uses a separate, dedicated evaluator that runs before generation begins and reacts with an external corrective action rather than the model reflecting on its own output. The option describing the model generating its own reflection tokens is therefore wrong, since that describes Self-RAG rather than CRAG's external evaluator. The option describing removing retrieval entirely for broad questions is wrong because CRAG's evaluator reacts to retrieval quality, not to whether a question is broad or narrow. The option describing re-embedding the whole corpus with a larger model on the fly is wrong because CRAG's corrective action is to seek additional or alternative sources such as web search, not to rebuild the existing index.
Source: Yan, Gu, Zhu & Ling, 'Corrective Retrieval Augmented Generation' (2024), arXiv:2401.15884