A team wants a support chatbot to answer questions about an internal policy document set that changes weekly. Instead of periodically fine-tuning the model on the updated documents, they build a system that retrieves the most relevant passages from a continuously re-indexed document store and inserts them into the prompt before the model generates its answer. What is the main advantage of this retrieval-augmented approach over repeatedly fine-tuning on the updated documents?
- The knowledge base can be kept current by re-indexing the changed documents alone, without retraining or replacing the model's weights, so newly added or edited information becomes available to the chatbot as soon as it is indexed
- Fine-tuning is incapable of teaching a model any new factual content, so repeating it on the updated documents would have no effect on the chatbot's answers at all
- Retrieval-augmented generation removes the model's context window limit entirely, so an unlimited number of documents can be inserted into every prompt
- The passages retrieved from the document store are automatically checked for factual accuracy before being shown to the model, which guarantees the chatbot's answers will never contain unsupported claims
Why A? And why not the others?
Correct answer: A. The knowledge base can be kept current by re-indexing the changed documents alone, without retraining or replacing the model's weights, so newly added or edited information becomes available to the chatbot as soon as it is indexed
Retrieval-augmented generation keeps the model's parameters untouched and instead updates a separate, continuously refreshed index of documents; because generation pulls whatever the retriever finds at query time, a policy change becomes usable the moment the changed document is re-indexed, with no retraining cycle in between, unlike fine-tuning, which requires collecting new training examples and running a training job every time the underlying documents change. The option claiming fine-tuning cannot teach a model any new facts overstates the case -- fine-tuning can shift a model's learned associations, it is simply slow, costly, and impractical to repeat every week, which is exactly why the comparison favors retrieval instead. The option describing an unlimited context window is wrong because retrieval augmentation still inserts retrieved text into a prompt that is bounded by the model's context window; that limit is precisely why only a small number of top-ranked passages, not the whole document set, are retrieved and inserted. The option promising guaranteed accuracy is wrong because retrieval only supplies the model with candidate passages; nothing about the retrieval step verifies the truth of those passages or forces the generated answer to stay faithful to them.
Source: Lewis et al., 'Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks' (2020), arXiv:2005.11401