A RAG system built on flat, similarity-ranked chunk retrieval answers narrow factual questions well but struggles with a question like 'what is the overall argument this document makes across all of its sections,' because no single chunk or small handful of chunks contains that overall picture. RAPTOR (Sarthi et al., 2024) was designed to address exactly this class of question. What does RAPTOR do differently from flat chunk retrieval?
- It fine-tunes the underlying language model directly on the document collection so broad questions can be answered from the model's own updated parameters instead of from any retrieval step
- It extracts named entities and their relationships from the corpus into a knowledge graph, then partitions that graph into communities and pre-generates a summary for each community
- It recursively embeds, clusters, and summarizes chunks from the bottom up, building a tree with multiple levels of summarization above the raw chunks, so a query can be answered from higher, more abstractive levels of the tree in addition to the original raw chunks, rather than being limited to whatever a single flat chunk happens to contain
- It increases the number of raw chunks retrieved for every query to the maximum the embedding model can accept in one batch, without adding any new structure above the chunks themselves
Why C? And why not the others?
Correct answer: C. It recursively embeds, clusters, and summarizes chunks from the bottom up, building a tree with multiple levels of summarization above the raw chunks, so a query can be answered from higher, more abstractive levels of the tree in addition to the original raw chunks, rather than being limited to whatever a single flat chunk happens to contain
RAPTOR builds a tree with differing levels of summarization from the bottom up by recursively embedding, clustering, and summarizing chunks, so that at query time the system can draw on higher, more abstractive levels of the tree that already synthesize many chunks' worth of content, instead of being limited to whatever a single flat chunk or a small handful of them happen to say; the paper reports a large absolute-accuracy gain on a complex-reasoning benchmark when this retrieval is combined with a strong generation model, specifically because broad questions benefit from these pre-built higher-level summaries. The option describing fine-tuning the language model on the corpus is wrong because RAPTOR keeps a retrieval-based architecture rather than baking corpus knowledge into model weights. The option describing extracting entities and relationships into a knowledge graph with community summaries describes a different technique (an entity-and-relationship graph approach), not RAPTOR's recursive clustering-and-summarization tree, which builds its hierarchy directly from chunk text and embeddings rather than from extracted entities. The option describing simply retrieving more raw chunks is wrong because it adds no new structure above the chunks and does nothing to synthesize information spread across many of them.
Source: Sarthi et al., 'RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval' (2024), arXiv:2401.18059