A RAG pipeline retrieves passages once, before generation begins, and then generates the entire answer from that single retrieved set. For a long, multi-part answer, the passages relevant to the answer's later sentences may be completely different from what was relevant to its first sentence, but the pipeline never retrieves again after that first pass. FLARE (Jiang et al., 2023) is designed to address this. How does it decide when to trigger a new retrieval step during generation?
- It retrieves again after every single generated token, regardless of how confident the model is in that token, to guarantee the freshest possible context throughout generation
- It waits until generation is completely finished, then retrieves once more to double check the finished answer, replacing the answer entirely if the second retrieval turns up different passages
- It asks a human reviewer to manually flag which sentences need additional retrieval before generation is allowed to continue past that point
- It uses its own prediction of the upcoming sentence to anticipate what that sentence will need, and if that anticipated sentence contains low-confidence tokens, it uses the anticipated content as a query to retrieve relevant documents and regenerates the sentence with that retrieved context, repeating this check throughout generation rather than retrieving only once upfront
Why D? And why not the others?
Correct answer: D. It uses its own prediction of the upcoming sentence to anticipate what that sentence will need, and if that anticipated sentence contains low-confidence tokens, it uses the anticipated content as a query to retrieve relevant documents and regenerates the sentence with that retrieved context, repeating this check throughout generation rather than retrieving only once upfront
FLARE iteratively predicts the upcoming sentence to anticipate what content it will need, and if that anticipated sentence contains low-confidence tokens, it treats the anticipated content as a query, retrieves relevant documents, and regenerates the sentence using that newly retrieved context; this check repeats across the course of generation rather than retrieving only once based on the original input, which is what lets it adapt to a long answer whose later sentences need different sources than its first sentence did. The option describing retrieving after every single token regardless of confidence is wrong because FLARE's trigger is specifically low-confidence tokens in the anticipated upcoming sentence, not an unconditional per-token retrieval. The option describing a single check-and-replace pass after generation finishes is wrong because FLARE's retrieval decisions happen throughout generation, sentence by sentence, not as one final verification step. The option describing a human reviewer manually flagging sentences is wrong because FLARE's retrieval trigger is fully automated, based on the model's own token-level confidence, with no human in that loop. This is a different concern from ordering already-retrieved passages within a single prompt (a lost-in-the-middle style problem) and from a coarser one-time decision about whether to retrieve at all, since FLARE's contribution is deciding when and how often to retrieve again mid-generation.
Source: Jiang, Xu, Gao, Sun, Liu, Dwivedi-Yu, Yang, Callan & Neubig, 'Active Retrieval Augmented Generation' (2023), arXiv:2305.06983