An engineer is building an agent that must look up information from a search API partway through solving a multi-step question and adjust its plan based on what the search returns. According to Yao et al. (2022), "ReAct: Synergizing Reasoning and Acting in Language Models," what does the ReAct prompting framework do to make this possible?
- Generating only the sequence of actions to take, such as API calls, without ever producing any intermediate natural-language reasoning
- Interleaving natural-language reasoning traces with task-specific actions and the observations those actions return, within a single prompted trajectory, so reasoning can decide the next action and each new observation can update the reasoning that follows
- Training a separate reasoning model and a separate acting model with reinforcement learning and combining their outputs after each has finished running independently
- Producing a chain-of-thought explanation for a problem but never issuing any call to an external tool or API
Why B? And why not the others?
Correct answer: B. Interleaving natural-language reasoning traces with task-specific actions and the observations those actions return, within a single prompted trajectory, so reasoning can decide the next action and each new observation can update the reasoning that follows
Yao et al. (2022) propose ReAct, which prompts a language model to generate reasoning traces and task-specific actions in an interleaved sequence within one trajectory: a reasoning step can plan or revise what to do next, an action step then queries an external source such as a search API, and the resulting observation is fed back into the next reasoning step, letting the model track and adjust its plan as new information arrives. The option describing actions with no reasoning describes the plain action-generation baseline ReAct is shown to outperform. The option describing separate reasoning and acting models trained independently with reinforcement learning describes a different architecture, not the single-model interleaved prompting method ReAct uses. The option describing reasoning with no external action describes plain chain-of-thought prompting, which cannot gather new information from outside the model the way ReAct's action steps do.
Source: Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (2022), arXiv:2210.03629