A team building a literary-translation tool has one LLM call produce a draft translation, then a second, separate LLM call critique that draft against specific criteria (tone, idiom, nuance) and return concrete feedback, which is fed back to the first call to produce a revised translation -- repeating this critique-and-revise cycle a few times before returning a final translation to the user. What Anthropic 'Building Effective Agents' pattern is this, and when is it worth the added latency?
- Routing -- classifying the translation request by source language and directing it to a specialized prompt per language, which is worth it whenever more than one language is supported
- Prompt chaining with no feedback loop -- a fixed sequence of translation subtasks that always runs the same way regardless of the draft's quality, worth it whenever a task can be split into steps
- Parallelization by voting -- running several independent translation attempts and picking whichever ones agree, worth it whenever a single translation call is considered unreliable
- Evaluator-optimizer -- one call generates a response while a separate call evaluates it and provides feedback in a loop, worth the added latency specifically when there are clear evaluation criteria and when iterative refinement based on that feedback provides measurable value, such as when a human's articulated feedback would visibly improve the response
Why D? And why not the others?
Correct answer: D. Evaluator-optimizer -- one call generates a response while a separate call evaluates it and provides feedback in a loop, worth the added latency specifically when there are clear evaluation criteria and when iterative refinement based on that feedback provides measurable value, such as when a human's articulated feedback would visibly improve the response
This is the evaluator-optimizer pattern: one LLM call generates a response while a separate call evaluates it and provides feedback in a loop, and it is worth its added latency specifically when there are clear evaluation criteria (here, tone, idiom, and nuance) and when iterative refinement driven by that feedback provides measurable value -- the kind of case where articulated feedback visibly improves the result, as with literary translation's need to capture nuance. The option describing routing is wrong because nothing here classifies the request and dispatches it to one specialized prompt; instead the same draft passes repeatedly between a generator and an evaluator. The option describing prompt chaining with no feedback loop is wrong because a fixed, always-the-same sequence of steps would not involve a separate call critiquing the draft and feeding concrete feedback back for revision -- the defining feature here is exactly that feedback loop, which plain sequential chaining lacks. The option describing parallelization by voting is wrong because voting runs multiple independent attempts at the same task simultaneously and combines them by agreement, whereas this scenario runs one draft through repeated sequential rounds of critique and revision rather than several independent attempts compared against each other.
Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes evaluator-optimizer as 'one LLM call generates a response while another provides evaluation and feedback in a loop,' appropriate 'when we have clear evaluation criteria, and when iterative refinement provides measurable value,' citing literary translation as a use case.