passdrill
AI & LLM Engineering · AI Agents & Tool Use · Card 016/022 easy

A support-bot pipeline embeds a user's question, retrieves the top-k matching documents from a vector store in a single lookup, and immediately passes those documents plus the question to the model to generate an answer, with no option to look anything up again. A different design instead lets the model itself decide, after seeing what a first retrieval turned up, whether the results are sufficient to answer or whether it should issue a further, refined query and retrieve again, repeating until it judges it has enough evidence. What is the key structural difference between these two designs?

  1. The first design is incapable of using more than one document at a time, while the second can only ever use exactly one document per query
  2. The first design performs a single, fixed retrieve-then-generate pass, while the second turns retrieval into an iterative control loop -- the model reasons about each retrieval's results and decides whether to stop or retrieve again with a refined query, at the cost of extra model calls and latency per additional round
  3. The second design removes the need for a vector store or any retrieval index at all, since the model can decide on its own what the correct answer is without looking anything up
  4. The two designs are functionally identical, since both eventually pass some retrieved text to the model before it generates an answer
Next card → Shuffle