A team wants an LLM to write and keep improving its own instruction for a reasoning task, given only a way to score any candidate instruction against a small held-out set (for example, accuracy on GSM8K). Zhou et al.'s Automatic Prompt Engineer (APE) generates a batch of candidate instructions in one pass from example input-output pairs and selects whichever scores best. Yang et al. (2023), "Large Language Models as Optimizers," propose OPRO for the same kind of problem. How does OPRO's procedure differ from APE's one-pass generate-and-select approach?
- OPRO trains a separate small neural network purely on the numeric scores to predict the best wording, without the LLM ever seeing the scores itself
- OPRO removes scoring from the loop entirely and instead has the LLM vote on which of several candidate instructions sounds most natural to a human reader
- OPRO runs an iterative optimization loop: at each step it feeds the model a "meta-prompt" containing the trajectory of previously tried instructions together with their scores, and asks the model to propose a new instruction intended to score higher than the ones already tried, so each new attempt is conditioned on the full history of earlier attempts rather than everything being generated in one pass
- OPRO requires updating the LLM's own weights via gradient descent on the scoring function, making it inapplicable to closed-weight models accessed only through an API
Why C? And why not the others?
Correct answer: C. OPRO runs an iterative optimization loop: at each step it feeds the model a "meta-prompt" containing the trajectory of previously tried instructions together with their scores, and asks the model to propose a new instruction intended to score higher than the ones already tried, so each new attempt is conditioned on the full history of earlier attempts rather than everything being generated in one pass
OPRO frames prompt discovery as an iterative optimization loop rather than a single generate-and-select pass: at every step, the model is shown a meta-prompt that lists the instructions tried so far alongside their measured scores, and it is asked to propose a new instruction that it expects to score higher than those already tried; that new instruction is scored, added to the trajectory, and the loop repeats, so later proposals are conditioned on the accumulated history of what has and has not worked, letting accuracy climb gradually starting from low-scoring initial prompts. This is a meaningfully different structure from APE, which produces a batch of candidate instructions in one pass from example input-output pairs and picks the best scorer without ever feeding scores back into a further round of generation. The option describing a separate neural network trained purely on scores is wrong because in OPRO the LLM itself reads the scores directly inside the meta-prompt and reasons over them in natural language. The option describing voting on naturalness is wrong because OPRO's proposals are selected by measured task performance, not by a naturalness preference. The option describing gradient descent on model weights is wrong because OPRO treats the optimizer LLM as a black box invoked through ordinary prompting, which is exactly why it works with closed, API-only models.
Source: Yang, Wang, Lu, Liu, Le, Zhou & Chen, 'Large Language Models as Optimizers' (arXiv:2309.03409, 2023)