Du et al. (2023), "Improving Factuality and Reasoning in Language Models through Multiagent Debate," test an alternative to having a single model instance revise its own answer alone, as in Self-Refine. What debate procedure do they propose, and how does it differ from a single model critiquing and refining its own output?
- A single model instance argues both sides of a debate against itself within one continuous response, then declares its own earlier argument the winner without ever comparing the two arguments against each other
- Multiple separate instances of a language model each independently produce an answer and its reasoning, then are shown one another's answers and reasoning over several rounds and asked to update their own response in light of the others' arguments, continuing until the instances converge on a shared final answer; this differs from Self-Refine, which uses one model instance for every role, generator, critic, and refiner, with no independent second party involved
- One model instance generates several candidate answers, and a much smaller, separately trained model is trained to grade which of those answers is factually best, with no back-and-forth exchange of arguments involved
- Multiple model instances are merged into a single set of weights via averaging before generating one answer, so no exchange of natural-language arguments occurs between separate active instances at inference time
Why B? And why not the others?
Correct answer: B. Multiple separate instances of a language model each independently produce an answer and its reasoning, then are shown one another's answers and reasoning over several rounds and asked to update their own response in light of the others' arguments, continuing until the instances converge on a shared final answer; this differs from Self-Refine, which uses one model instance for every role, generator, critic, and refiner, with no independent second party involved
Multiagent debate has multiple separate instances of a language model, or of several different models, each independently generate their own answer and reasoning for the same question, then exposes each instance to the other instances' answers and reasoning over multiple rounds, asking every instance to update its own response in light of what the others argued; the process continues across rounds until the instances converge on a shared final answer, which the paper reports improves factual accuracy and reduces hallucinated or fallacious answers compared to a single instance working alone. This differs from Self-Refine specifically because Self-Refine uses exactly one model instance to play every role, generating a draft, critiquing it, and refining it, with no second, independently reasoning party ever involved, whereas debate depends on genuinely separate instances that can disagree with each other. The option describing one instance arguing both sides internally is wrong because it never actually confronts an independent second perspective, unlike true multiagent debate. The option describing a separately trained grading model with no argument exchange is wrong because debate's core mechanism is the multi-round exchange of reasoning between instances, not a one-shot grading step. The option describing weight averaging before generation is wrong because debate operates entirely at inference time through natural-language exchange, never merging the models' parameters together.
Source: Du, Li, Torralba, Tenenbaum & Mordatch, 'Improving Factuality and Reasoning in Language Models through Multiagent Debate' (arXiv:2305.14325, 2023)