Deng et al. (2023), "Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves," start from the observation that a human's phrasing of a question often carries ambiguity or missing context that a language model reads differently than the human intended. What does the Rephrase and Respond (RaR) method have the model do about this, and how does the paper describe its relationship to chain-of-thought prompting?
- RaR trains a separate classifier to detect ambiguous questions and reroutes only those questions to a human reviewer before any model response is generated
- RaR has the model rephrase and expand the given question in its own words, adding clarifying detail, before answering, which the paper shows is complementary to chain-of-thought rather than a replacement for it, and combining the two performs better than either alone
- RaR instructs the model to translate the question into a different natural language first, on the theory that translation removes ambiguity, and this fully replaces the need for chain-of-thought
- RaR skips rephrasing entirely and instead asks the model to answer the same question multiple times, then rephrases only the final selected answer for readability
Why B? And why not the others?
Correct answer: B. RaR has the model rephrase and expand the given question in its own words, adding clarifying detail, before answering, which the paper shows is complementary to chain-of-thought rather than a replacement for it, and combining the two performs better than either alone
Rephrase and Respond has the model itself rephrase and expand the original question, adding clarifying detail that resolves the gap between how a human framed the question and how the model would ideally like it framed, before producing an answer to its own rephrased version. The paper explicitly compares RaR to chain-of-thought both theoretically and empirically and finds the two are complementary rather than competing techniques, with combining rephrasing and step-by-step reasoning outperforming either technique used alone. The option describing a separate ambiguity-detecting classifier that routes questions to a human reviewer is wrong because RaR involves no human intervention or classifier at all; the same model handles rephrasing and answering end to end. The option describing literal translation into another language is wrong both because that is not the mechanism RaR uses and because the paper frames RaR as complementary to chain-of-thought, not as something that fully replaces it. The option describing multiple answer attempts with only the final answer rephrased is wrong because RaR rephrases the question before answering, not the answer after the fact.
Source: Deng, Zhang, Chen & Gu, 'Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves' (arXiv:2311.04205, 2023)