A user asks a RAG system, 'How did the pricing model change between the 2023 and 2024 versions of the product, and which change had the bigger effect on enterprise customers?' A single embedding search against this entire question, as written, tends to retrieve passages that are each only partially relevant, because the question actually bundles together more than one distinct piece of information the retriever needs to find. What technique addresses this, and how does it work?
- Increasing k, the number of passages retrieved, so that even though each individual retrieved passage is only partially relevant, enough of them are returned that the full answer is guaranteed to be present somewhere in the larger set
- Switching the embedding model to one with a larger number of dimensions, since more dimensions let a single vector capture every distinct piece of information a compound question could bundle together
- Applying a stricter similarity-score cutoff to the single search, which removes only the weakest partial matches and leaves just the passages that are relevant to the whole compound question
- Query decomposition: breaking the original compound question into separate, narrower sub-questions (such as one about the 2023-to-2024 pricing change and one about which change affected enterprise customers more), retrieving separately for each sub-question, and then combining the retrieved evidence when generating the final answer
Why D? And why not the others?
Correct answer: D. Query decomposition: breaking the original compound question into separate, narrower sub-questions (such as one about the 2023-to-2024 pricing change and one about which change affected enterprise customers more), retrieving separately for each sub-question, and then combining the retrieved evidence when generating the final answer
The question above actually asks for two distinct things -- what changed about pricing between two versions, and which of those changes mattered more to one customer segment -- and a single embedding of the whole sentence produces one vector that blends both asks together, which tends to retrieve passages that partially match one part or the other rather than passages that fully match either; query decomposition fixes this by splitting the compound question into narrower sub-questions first, retrieving separately for each one so that each search targets a single, coherent piece of information, and only combining the separately retrieved evidence at generation time. Simply increasing the number of retrieved passages does not fix the underlying problem, since the single blended query embedding is still what determines which passages are considered close matches in the first place; adding more of the same imprecisely matched candidates does not guarantee the missing, more precisely relevant ones appear. Switching to a higher-dimensional embedding model does not solve this either, since more dimensions increase how much information a vector can represent in general but do not give a single embedding of a multi-part question the ability to separately match multiple distinct pieces of information at once. Applying a stricter similarity cutoff on the original single search only removes the weakest matches from an already poorly targeted result set; it cannot introduce the more precisely relevant passages that a differently phrased, narrower query would have found.
Source: NVIDIA, 'Query Decomposition for NVIDIA RAG Blueprint,' https://docs.nvidia.com/rag/latest/query_decomposition.html; 'Question Decomposition for Retrieval-Augmented Generation' (2025), arXiv:2507.00355