A model asked to write a short biographical paragraph about a real historical figure produces a mostly accurate response but invents two incorrect dates. Dhuliawala et al. (2023), "Chain-of-Verification Reduces Hallucination in Large Language Models," propose a four-step prompting procedure (CoVe) meant to catch this kind of error before the response is delivered to the user. What are the four steps, in order?
- Draft an answer, then immediately ask the model to rate its own confidence on a 1-10 scale, then deliver whichever draft scores highest without any further steps
- Draft an initial response, plan a set of targeted verification questions that would fact-check specific claims in that draft, answer each verification question independently so those answers are not biased by the original draft, then use the verification answers to produce a final, revised response
- Draft an answer, translate it into a different language and back, compare the two versions for discrepancies, and keep whichever version is shorter
- Draft an answer, retrieve external documents matching every named entity in the draft, replace every named entity with whatever the retrieved documents say without any further verification step, and stop
Why B? And why not the others?
Correct answer: B. Draft an initial response, plan a set of targeted verification questions that would fact-check specific claims in that draft, answer each verification question independently so those answers are not biased by the original draft, then use the verification answers to produce a final, revised response
Chain-of-Verification proceeds in four steps: the model drafts an initial response to the query, then plans a set of verification questions targeted at fact-checking specific claims made in that draft, then answers each verification question independently of the original draft so those answers are not biased by whatever the draft already claimed, and finally uses the verification answers to produce a final response that corrects any discrepancies the verification step surfaced. The paper shows this reduces hallucinations across tasks including list-based questions from Wikidata, closed-book MultiSpanQA, and longform text generation, precisely the kind of invented-date error in the biography scenario. The option describing a simple self-rated numeric confidence score is wrong because it skips the independent verification-question step entirely and does not fact-check specific claims. The option describing round-trip translation is wrong because no translation step appears anywhere in CoVe. The option describing blind replacement of named entities from retrieved documents with no independent verification step is wrong because CoVe's verification relies on the model answering its own generated verification questions, not on an unverified retrieval-based substitution.
Source: Dhuliawala et al., 'Chain-of-Verification Reduces Hallucination in Large Language Models' (arXiv:2309.11495, 2023)