A support-bot pipeline embeds a user's question, retrieves the top-k matching documents from a vector store in a single lookup, and immediately passes those documents plus the question to the model to generate an answer, with no option to look anything up again. A different design instead lets the model itself decide, after seeing what a first retrieval turned up, whether the results are sufficient to answer or whether it should issue a further, refined query and retrieve again, repeating until it judges it has enough evidence. What is the key structural difference between these two designs?
- The first design is incapable of using more than one document at a time, while the second can only ever use exactly one document per query
- The first design performs a single, fixed retrieve-then-generate pass, while the second turns retrieval into an iterative control loop -- the model reasons about each retrieval's results and decides whether to stop or retrieve again with a refined query, at the cost of extra model calls and latency per additional round
- The second design removes the need for a vector store or any retrieval index at all, since the model can decide on its own what the correct answer is without looking anything up
- The two designs are functionally identical, since both eventually pass some retrieved text to the model before it generates an answer
Why B? And why not the others?
Correct answer: B. The first design performs a single, fixed retrieve-then-generate pass, while the second turns retrieval into an iterative control loop -- the model reasons about each retrieval's results and decides whether to stop or retrieve again with a refined query, at the cost of extra model calls and latency per additional round
A fixed, single-shot retrieval pipeline retrieves once and generates once, with no mechanism for the model to notice that what it got back was insufficient and act on that; the alternative -- often called agentic RAG -- turns retrieval into a control loop where the model evaluates each retrieval's results and decides whether to stop or issue another, more targeted query, which lets it recover from a first retrieval that missed the mark, but each extra round costs another model call and added latency. The option limiting either design to exactly one document per query is wrong: top-k retrieval in the fixed design already returns multiple documents in one pass, and nothing about the iterative design limits a single query to one document either; the difference is whether retrieval can repeat, not how many documents come back per call. The option claiming the iterative design removes the need for a retrieval index is wrong: the model is still querying the same kind of vector store or index, just potentially more than once and with revised queries. The option calling the two designs functionally identical is wrong because it erases exactly the distinction being tested: one design cannot revisit a bad first retrieval, and the other can.
Source: NVIDIA Technical Blog, 'Traditional RAG vs. Agentic RAG -- Why AI Agents Need Dynamic Knowledge to Get Smarter' (developer.nvidia.com/blog/traditional-rag-vs-agentic-rag-why-ai-agents-need-dynamic-knowledge-to-get-smarter) -- describes traditional RAG's fixed retrieve-generate flow versus agentic RAG's dynamic retrieve/reason/decide control loop