A customer-facing chatbot uses two separate, simultaneously-running LLM calls on every incoming user message: one call's only job is to screen the message for inappropriate or out-of-policy content, while a completely separate call independently drafts the actual response to the user's question; the two calls' outputs are then combined programmatically before anything is shown to the user. Why does Anthropic's guidance describe handling guardrails this way, rather than having the single call that drafts the response also perform its own content screening?
- Because this is an application of the parallelization pattern's sectioning variation, and separating the screening concern into its own dedicated call tends to perform better than having one call juggle both the core response and the guardrail check at the same time
- Because a single LLM call is technically incapable of ever producing more than one type of output, so screening and drafting could never both happen within one call regardless of design
- Because running two calls is always cheaper than running one call, so splitting any task into two simultaneous calls reduces cost regardless of what each call does
- Because Anthropic's documentation states that content screening must legally be performed by a human reviewer, and no LLM call of any kind is permitted to perform it
Why A? And why not the others?
Correct answer: A. Because this is an application of the parallelization pattern's sectioning variation, and separating the screening concern into its own dedicated call tends to perform better than having one call juggle both the core response and the guardrail check at the same time
This is a guardrails implementation of the parallelization pattern's sectioning variation: the task is split into independent pieces run simultaneously, one instance handling content screening and a separate instance handling the actual response, and Anthropic's guidance notes this tends to perform better than having the same LLM call handle both guardrails and the core response at once. The option claiming a single call is technically incapable of producing more than one type of output is wrong -- nothing prevents one call from attempting both screening and drafting together; the reason to separate them is that doing so tends to perform better, a quality argument, not a hard technical impossibility. The option claiming two calls are always cheaper than one is wrong and backwards: running two simultaneous calls instead of one generally costs more, not less, so cost reduction is not the reason for this design. The option claiming content screening must legally be performed by a human reviewer is wrong and unsupported -- nothing in this guidance requires human review of every message; the screening call described here is itself an LLM call, not a human one.
Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes guardrails as a sectioning application of parallelization: 'one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.'