AI Agents & Tool Use

Correct answer: .

The model has no ability to execute code, open network connections, or otherwise act on external systems; it can only produce structured content describing which tool it wants used and with what arguments. Execution happens in the calling application's own code, which parses that structured request, runs the corresponding function against the real system, and returns the function's output back into the conversation as a new message so the model can use it to keep responding. The option describing the model executing the action directly is wrong because generating a tool-use request and actually performing an operation are different capabilities, and only the second requires access the model does not have. The option describing the target system's own API silently intercepting the response is wrong because nothing about a ticketing system's API watches a model's raw output; the calling application must explicitly contain the code that recognizes the request and invokes the matching function. The option claiming models are barred from naming tools is wrong because naming an available tool and its arguments is exactly the mechanism this architecture relies on; without it the calling application would have nothing to execute.

Source: Anthropic, 'Tool use with Claude' documentation (platform.claude.com/docs/en/agents-and-tools/tool-use/overview) -- describes the general client-tool execution model (model requests, application executes) shared across major vendor function-calling APIs

Correct answer: .

This Thought/Action/Observation pattern is the ReAct approach: interleaving reasoning traces with actions lets the model's later reasoning steps update on real observations pulled from an external environment (such as a knowledge-base query), rather than continuing to build on an earlier reasoning step that may already have drifted from the facts. Yao et al. showed this measurably reduces the hallucination and error propagation that plain chain-of-thought reasoning is prone to on multi-step question-answering and fact-verification tasks, because a wrong intermediate claim gets corrected by an observation instead of being compounded further. The option claiming complete freedom from factual errors overstates the benefit -- retrieved sources can themselves be wrong or misread, so grounding reduces but does not eliminate error. The option describing reasoning as removed entirely misunderstands the pattern, since the Thought step is reasoning and remains essential to deciding what to look up next. The option about a permanently larger context window confuses a prompting pattern with a model's architecture; interleaving Thought/Action/Observation does not change how many tokens the underlying model can attend to.

Source: Yao et al., 'ReAct: Synergizing Reasoning and Acting in Language Models' (2022), arXiv:2210.03629

Correct answer: .

Agent loops that repeatedly observe, decide, and act need an explicit stopping rule built into the loop itself; frameworks such as AutoGen make this concrete with settings like a maximum number of consecutive automatic replies and a check for a recognizable termination message, either of which ends the loop even if the underlying task was never actually solved. Without some such condition, an agent that keeps generating plausible-looking next actions has no built-in reason to stop, and will keep looping even when it is only repeating a failed attempt. The option pointing to too few tools is wrong because the number of available tools has nothing to do with whether the loop recognizes it should stop -- even a single-tool agent needs a stopping rule. The option pointing to a small context window is wrong because forgetting earlier attempts might make an agent repeat itself, but expanding the window is not what supplies a stopping condition, and plenty of agents loop indefinitely with plenty of room left in context. The option invoking retrieval-augmented generation is wrong because RAG is about pulling in external knowledge to ground answers, and has no special role in judging whether a task is complete.

Source: Wu et al., 'AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework' (2023), arXiv:2308.08155 -- describes explicit conversation termination conditions (e.g. a cap on consecutive automatic replies, and a check for a termination message) required to stop an agent loop

Correct answer: .

Because the code being run was generated by a model that can be wrong, manipulated by injected instructions in the data it processes, or simply buggy, isolating execution in a sandboxed container with restricted resources and no path to production systems limits how much damage any single generated command can do, regardless of why it went wrong. This is exactly why dedicated code-execution tools for agents describe themselves as running Python and bash in a sandboxed container rather than on the host directly. The option claiming models cannot produce valid code outside a container is wrong -- a model can generate syntactically correct code whether or not that code is ever executed, and containers do not affect what a model is capable of writing. The option claiming a sandbox rewrites unsafe commands into safe ones is wrong because isolation does not modify the code; it only limits what the code, however written, is able to reach and affect. The option about raw execution speed is wrong because sandboxing typically adds overhead rather than removing it; the point of the isolation is containment of risk, not performance.

Source: Anthropic, 'Tool use with Claude' documentation (platform.claude.com/docs/en/agents-and-tools/tool-use/overview), 'Code execution tool' entry: 'Run Python and bash code in a sandboxed container to analyze data and generate files'

Correct answer: .

Exposing only a fixed set of named tools, each with its own description and input schema, is how an agent's available actions are scoped at the request level -- the model chooses from whatever list of tools it was given, so an operation like issuing a refund, which was never defined or described to it, was never presented as something it could ask for. This is a design-time restriction on what gets offered, not a guarantee about what the model will output. The option claiming the model literally cannot generate text describing an unlisted operation is wrong, because a model can still produce free-form text or even attempt a malformed request referencing something outside the defined set; scoping the tool list reduces the chance of this but the calling application still has to check what comes back. The option claiming the calling application no longer needs to verify incoming requests is wrong for the same reason -- a smaller tool list lowers risk but does not remove the need to validate that a request actually matches a defined tool before running anything. The option claiming the tool list has no effect on what the agent can request is wrong because restricting the exposed tools is precisely what constrains the space of actions the model is prompted to choose among.

Source: Anthropic, 'Tool use with Claude' documentation (platform.claude.com/docs/en/agents-and-tools/tool-use/overview), 'Define tools' entry: 'Specify tool schemas, write descriptions, and control when Claude calls your tools'

Correct answer: .

Human-in-the-loop approval gates are placed at the specific steps where a mistake would be costly and hard to undo, which is exactly the distinction drawn between drafting a refund and actually issuing one: drafting can be revised or discarded with no real-world effect, but issuing moves real money and may be difficult or impossible to fully claw back, so it is that step, not the earlier read-only or draft steps, that gets a pause for explicit confirmation. Frameworks that support this kind of workflow let a human's input be required only for certain messages (for instance, ones that look like a completed, ready-to-execute action) rather than for every step the agent takes. The option claiming models are barred from drafting a refund amount at all is wrong, since drafting is explicitly the ungated step in this design. The option treating inventory checks and refund issuance as technically identical is wrong because they differ exactly on the dimension that matters here -- reversibility and financial consequence -- which is why only one of them is gated. The option claiming the payment tool cannot technically be called by an agent is wrong; the whole point of the design is that it can be called, which is precisely why an approval step is placed in front of it.

Source: Wu et al., 'AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework' (2023), arXiv:2308.08155 -- describes configuring human input to be requested at specific points in an agent's conversation rather than at every turn

Correct answer: .

Decoupling planning from execution -- as in the ReWOO approach's Planner, Worker, and Solver split -- lets the reasoning model produce a full blueprint of what lookups are needed just once, hand the actual tool calls to a separate lightweight component, and only bring the reasoning model back in to compose the final answer, instead of re-feeding the whole accumulating transcript of past thoughts and observations into the expensive reasoning model after every single lookup. Xu et al. report this cuts token consumption substantially compared to designs that interleave reasoning and tool calls at every step, while also holding up better when a tool call fails, since the plan does not have to be rebuilt from scratch. The option calling this the only way to use more than one tool is wrong, since interleaved reasoning-and-acting designs also chain multiple tool calls; the difference here is efficiency and structure, not tool count. The option claiming guaranteed higher accuracy on every task is wrong, since decoupling planning from execution can lose the benefit of adjusting the plan mid-task when an early result changes what is actually needed. The option claiming lookups can never fail or return something unusable is wrong -- the design is reported to be more robust to tool failure, not immune to it.

Source: Xu et al., 'ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models' (2023), arXiv:2305.18323

Correct answer: .

As an agent's history of observations and actions grows across many steps, that full history quickly exceeds what a single prompt can hold, and re-sending an ever-growing transcript on every turn would also be wasteful even where it does fit; the memory-stream/reflection/retrieval architecture Park et al. describe addresses this by storing the raw log and higher-level reflections outside the prompt entirely and pulling back only the entries judged relevant to the current situation, rather than the whole history. The option equating this with retrieval-augmented generation over documents is wrong -- RAG typically retrieves external factual/reference content, whereas this design retrieves the agent's own past experience and self-generated reflections, which is a related but distinct use of retrieval. The option claiming models retain nothing at all between turns within a single ongoing conversation is wrong; within the bounds of one context window a model does see prior turns directly, and the memory system exists specifically because that direct visibility does not scale indefinitely. The option claiming reflections gate every action regardless of simplicity is wrong -- reflections are generated periodically from accumulated observations, not required as a precondition for each individual action.

Source: Park et al., 'Generative Agents: Interactive Simulacra of Human Behavior' (2023), arXiv:2304.03442

Correct answer: .

The defining feature of a multi-agent design like this is splitting a task across separate, specialized conversable agents -- one for planning, one for generating a solution, one for reviewing it -- that pass messages to each other rather than a single agent handling every role internally in one uninterrupted stream of reasoning; frameworks such as AutoGen build agents specifically to converse with each other in this way, mixing roles and even combinations of models, human input, and tools per agent. The option restricting tool use to only one agent in the group is wrong -- nothing about a multi-agent design requires that only one participant may call tools, and different agents can each have their own tools. The option requiring every agent to share the exact same system prompt is wrong and actually contradicts the point of specialization -- giving each agent a distinct role-specific prompt is what lets the planner, coder, and reviewer behave differently from one another. The option claiming no termination condition is needed is wrong; a multi-agent conversation still needs an explicit stopping point (such as the reviewer signalling approval or a reply-count limit), or it risks looping the same way a single-agent loop can.

Source: Wu et al., 'AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework' (2023), arXiv:2308.08155

Correct answer: .

Content that arrives through a tool -- a fetched webpage, a search result, a document -- is data the agent was asked to process, not a second source of instructions with the same authority as the user who set up the task; treating anything appearing in context as an obeyable command is exactly the vulnerability that indirect prompt injection exploits, where an attacker plants instruction-like text inside content they know an LLM-integrated application will retrieve and feed to the model. The correct handling is to read and summarize the fetched text while refusing to let it redirect the agent into an unrelated action, such as exfiltrating credentials, that the actual user never requested. The option treating fetched content as a new authoritative instruction is wrong because that is precisely the failure mode being described, not the defense against it. The option saying fetched content can simply be ignored is wrong because the whole point of the tool is that its content is needed to complete the user's original request (summarizing the page); ignoring it defeats the task. The option treating any mentioned email address as an automatic trigger to send a message is wrong because it would let the mere appearance of text inside untrusted content directly cause a real-world action, which is the same failure mode restated rather than a solution to it.

Source: Greshake et al., 'Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection' (2023), arXiv:2302.12173

Correct answer: .

Because a model's tool-use request is generated text and not a guaranteed-correct reference to what was actually defined, it can name a tool that does not exist or supply arguments that do not match the declared schema -- this is a real failure mode, which is exactly why some tool-use implementations offer a strict mode specifically to make requests conform to the declared schema, implying that conformance is not otherwise guaranteed. The safe response to an unrecognized tool name or a schema mismatch is for the calling application to refuse to execute anything against real systems and instead return an error back to the model as an observation, letting the model correct itself and retry with a valid request. The option of executing against the closest-matching real tool is wrong because guessing at a destructive or unintended action (here, one that also deletes drafts) from a malformed request is far riskier than simply refusing and asking the model to retry. The option of silently renaming fields to match the schema is wrong for the same reason -- silently reinterpreting a request assumes an intent that was never actually validated. The option of extending the real tool's behavior at runtime to match the hallucinated request is wrong because it lets an unverified, possibly hallucinated capability (deleting drafts) actually happen, which is the opposite of containing the mismatch safely.

Source: Anthropic, 'Tool use with Claude' documentation (platform.claude.com/docs/en/agents-and-tools/tool-use/overview) -- 'Handle tool calls' entry: 'Parse tool_use blocks, format tool_result responses, and handle errors'; and 'Strict tool use' feature description noting schema conformance is not otherwise guaranteed

Correct answer: .

In Claude's tool-use API, when Claude decides to call a defined tool it returns a content block marking the request along with a unique identifier for that call; the application executes the corresponding function and then sends the result back inside a new message as a content block explicitly marked as the tool's result, carrying that same identifier so Claude can match the result to the specific call it made, even if multiple tool calls were requested at once. The option describing an unstructured text append with no identifier is wrong because Claude's API relies on that explicit identifier match, not on message ordering alone, to associate a result with its originating request. The option describing a modification to the original tool-definition request is wrong because the result is sent as part of the ongoing conversation's messages in a follow-up request, not as an edit to the tool definitions themselves, which stay fixed. The option claiming tool results must appear as a system-level instruction outside the conversation is wrong because the documented round trip places the tool's result inside the message list alongside the user's and Claude's own turns, not in a separate system-level channel.

Source: Anthropic, 'Tool use with Claude' documentation (platform.claude.com/docs/en/agents-and-tools/tool-use/overview): 'the response carries a tool_use block, your code runs the lookup, and a second request sends the result back in a tool_result block'

Correct answer: .

By default a single assistant turn can contain several tool_use blocks when the requested tools are independent; the API does not mandate a fixed execution order for multiple calls in one turn -- concurrent, sequential, or any mix is acceptable depending on what the tools do -- but whatever order is chosen, every tool_use block must get a matching tool_result in the same next user message, tied together by tool_use_id so Claude can attribute each result correctly. The option describing silently discarding extra blocks is wrong: nothing in the API drops additional tool_use blocks, and ignoring one would leave that request unanswered. The option describing merging both calls' arguments into one combined call is wrong: each tool_use block names a specific, separately defined tool with its own arguments, and conflating them would corrupt both requests, since the two tools take entirely different parameters. The option describing waiting for a second follow-up request is wrong: batching multiple tool_result blocks into a single message is exactly what the API supports, so there is no need to split them across separate round trips.

Source: Anthropic, 'Parallel tool use' (platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use): 'the response...can contain several tool_use blocks in a single assistant turn'; 'return one tool_result for each tool_use block...Match each result to its call with tool_use_id'

Correct answer: .

MCP standardizes the connection between an AI application (a host) and an external system's capabilities (a server), so that instead of writing a bespoke integration for every agent-to-tool pairing, a system exposes its capabilities once as an MCP server and any MCP-compatible host can connect to it through a client, cutting many bespoke one-off integrations down to shared servers plus hosts speaking one common protocol. The option describing removed security boundaries is wrong: MCP's architecture explicitly maintains clear security boundaries and isolates each server so it cannot read the whole conversation or see into other servers, the opposite of removing checks. The option describing one shared server merging every capability is wrong: the architecture's own model gives each client a 1:1 relationship with a particular server, precisely so the Git, database, and ticketing capabilities can stay as separate, independently composable servers rather than being merged into one. The option limiting an agent to one tool call per client is wrong: nothing about the client-server connection topology caps how many tool calls a client can make to its server.

Source: Model Context Protocol specification, 'Architecture' (modelcontextprotocol.io/specification/2026-07-28/architecture): 'client-host-server architecture where each host can run multiple client instances'; 'each client having a 1:1 relationship with a particular server'; 'Servers should not be able to read the whole conversation, nor see into other servers'

Correct answer: .

Tree of Thoughts generalizes chain-of-thought-style approaches by treating problem solving as a search over a tree whose nodes are intermediate 'thoughts', generating multiple candidate thoughts at a decision point, using the model itself to heuristically judge how promising each one looks, and applying a standard search procedure like breadth-first or depth-first search to explore and backtrack across branches -- exactly the capability ReAct's single committed Thought/Action/Observation line lacks, since ReAct never revisits or abandons an earlier step once it has moved past it. The option about removing intermediate reasoning text entirely is wrong: the 'thoughts' being searched over are themselves coherent units of reasoning text, not a bare numeric score replacing them. The option requiring every branch to call a real external tool is wrong: this search happens over the model's own generated reasoning candidates and evaluations, and does not depend on tool calls the way agent-environment interaction loops do. The option guaranteeing the globally best solution on the first attempt is wrong: a heuristic evaluation can still misjudge a branch's promise, so the search improves the chance of finding a good solution without guaranteeing the optimal one.

Source: Yao et al., 'Tree of Thoughts: Deliberate Problem Solving with Large Language Models' (2023), arXiv:2305.10601 -- frames problem solving as search over a tree of thoughts using BFS/DFS with heuristic evaluation, contrasted with left-to-right token-level decision-making

Correct answer: .

A fixed, single-shot retrieval pipeline retrieves once and generates once, with no mechanism for the model to notice that what it got back was insufficient and act on that; the alternative -- often called agentic RAG -- turns retrieval into a control loop where the model evaluates each retrieval's results and decides whether to stop or issue another, more targeted query, which lets it recover from a first retrieval that missed the mark, but each extra round costs another model call and added latency. The option limiting either design to exactly one document per query is wrong: top-k retrieval in the fixed design already returns multiple documents in one pass, and nothing about the iterative design limits a single query to one document either; the difference is whether retrieval can repeat, not how many documents come back per call. The option claiming the iterative design removes the need for a retrieval index is wrong: the model is still querying the same kind of vector store or index, just potentially more than once and with revised queries. The option calling the two designs functionally identical is wrong because it erases exactly the distinction being tested: one design cannot revisit a bad first retrieval, and the other can.

Source: NVIDIA Technical Blog, 'Traditional RAG vs. Agentic RAG -- Why AI Agents Need Dynamic Knowledge to Get Smarter' (developer.nvidia.com/blog/traditional-rag-vs-agentic-rag-why-ai-agents-need-dynamic-knowledge-to-get-smarter) -- describes traditional RAG's fixed retrieve-generate flow versus agentic RAG's dynamic retrieve/reason/decide control loop

Correct answer: .

As an agent loop runs for longer, the transcript of everything it has read, done, and observed keeps accumulating within the same context window, and unlike a fixed document that starts and stays a known size, this working transcript has no natural ceiling short of the task simply ending -- so a harness that wants to keep going past the point a raw transcript would overflow the window needs to actively curate what stays in full context versus what gets condensed into a shorter summary, treating the window as a limited resource to be filled with the highest-signal information rather than an ever-growing raw log. The option claiming the model can only read a few hundred tokens of any output is wrong and describes a much smaller, fixed limit than modern context windows actually have; the real problem is the never-ending growth of an ongoing session, not a tiny per-call cap. The option claiming every tool call must run twice is wrong and unrelated to transcript size. The option tying this to retrieval-augmented generation is wrong: condensing a transcript that already exists in-session is a different mechanism from retrieving external documents, and nothing about giving an agent multiple tools requires RAG.

Source: Anthropic, 'Effective context engineering for AI agents' (anthropic.com/engineering/effective-context-engineering-for-ai-agents) -- describes treating the context window as a finite resource, agents summarizing completed work and curating the highest-signal tokens as sessions grow long

Correct answer: .

Evaluating only the final outcome tells you whether a run ended well, but not how it got there, so an agent that skipped a required check and still happened to land on the right refund decision would score identically to one that actually verified eligibility properly; inspecting the trajectory -- the sequence of tool calls, their order, and whether required steps were actually taken -- surfaces that kind of hidden failure even when the end result looks fine, which is exactly the gap outcome-only evaluation misses. The option claiming the final outcome no longer needs checking is wrong: a correct sequence of steps still doesn't guarantee a correct outcome if, say, a tool itself returns bad data, so both levels remain useful together rather than one replacing the other. The option limiting trajectory evaluation to single-tool agents is wrong: examining an ordered sequence of steps works precisely because there are multiple tool calls to look at; it isn't blocked by having more than one. The option claiming guaranteed agreement with a human reviewer is wrong: any automated evaluation, trajectory-based or not, can still diverge from human judgment on a given run.

Source: LangChain, 'LLM Evaluation Framework: Trajectories vs. Outputs' (langchain.com/resources/llm-evaluation-framework) -- distinguishes trajectory-level evaluation of an agent's full sequence of tool calls and intermediate steps from outcome-level evaluation of only the final result

Correct answer: .

Defining only one tool scopes what the model is offered to ask for, but it doesn't by itself limit what that one tool is capable of doing once a request reaches the real system; scoping the credential behind the tool to read-only access on just the needed tables adds a second, independent layer of containment, so that even a call that was somehow malformed, manipulated, or wrongly generated still can't write data or reach unrelated tables, because the credential itself lacks that permission regardless of what the request asked for. The option claiming the credential is irrelevant once a tool is defined is wrong and is exactly the gap this defense-in-depth layer closes -- a defined tool can still be misused unless its actual access is separately capped. The option claiming a model literally cannot generate a write-style request is wrong: a model can still produce text requesting an action beyond what its credential allows; the credential is what blocks that action from succeeding, not what stops the model from asking. The option claiming schema validation becomes unnecessary is wrong: a scoped credential restricts what the query can affect once it runs, but it doesn't check whether the call's structure matches the tool's expected arguments in the first place, so validation is still a separate, still-needed step.

Source: OWASP Gen AI Security Project, LLM06:2025 Excessive Agency (genai.owasp.org/llmrisk) -- recommends per-tool permission scoping (read-only vs. write, specific resources) as a mitigation distinct from limiting which tools are exposed

Correct answer: .

An idempotency key lets a caller mark a request as representing one specific logical action rather than a fresh one; when the same key is attached to a retried call, a provider that supports this checks whether it has already completed a request under that key and, if so, returns the original result instead of executing the operation again -- which is exactly the protection needed here, since the agent's retry was triggered by a lost response to a charge that had, in fact, already succeeded, and without this mechanism the customer would be billed twice for a single intended action. The option claiming idempotency keys only affect logging is wrong: their entire purpose is to change execution behavior on the provider's side, skipping re-execution rather than merely recording that a repeat happened. The option claiming the retry never reaches the provider at all is wrong: the deduplication check described here happens on the provider's side once the request arrives, not by the calling code silently refusing to send it. The option describing an automatic refund of the first charge is wrong and inverts the mechanism: the provider keeps and returns the result of the original successful charge rather than treating it as invalid.

Source: General API idempotency-key pattern as applied to agentic tool retries (e.g. TianPan.co, 'The Idempotency Problem in Agentic Tool Calling'; 'Idempotent AI Agents: Retry-Safe Patterns for Production') -- describes attaching a stable key to a request so a provider that sees the same key twice returns the original result instead of re-executing the side effect

Correct answer: .

A group-chat-style design lets every participating agent read and respond to what every other agent in the conversation says, so a planner, coder, and reviewer can react to each other's messages directly within one shared thread; an orchestrator/lead-and-subagents design instead keeps subagents isolated from each other entirely -- each independently works its own piece of the task and reports back only to the lead agent, which is the sole point where results are combined -- trading the possibility of subagents catching or building on each other's work directly for a simpler, more predictable, strictly hierarchical coordination structure. The option limiting the first design to a single tool across the whole conversation is wrong: nothing about agents conversing directly with each other restricts how many tools are available across the group; each participant can still have its own tools. The option calling the two designs functionally identical is wrong because it erases the exact distinction being tested -- peer-to-peer visibility versus strict hierarchy with no lateral communication. The option requiring identical prompts and roles for every subagent is wrong: subagents in the hierarchical design can be given distinct subtasks and instructions, and the lead agent's synthesis step works specifically because each one reports back on its own distinct piece of the work.

Source: Anthropic, 'How we built our multi-agent research system' (anthropic.com/engineering/multi-agent-research-system) -- describes an orchestrator-worker pattern where subagents operate in parallel, do not communicate with each other, and report findings back to a LeadResearcher which synthesizes results

Correct answer: .

Because a model chooses and fills in tool calls based on the names, descriptions, and parameter documentation it is given, vague or ambiguous wording -- a tool named `f1` described only as 'does the thing' -- gives it little to work with when deciding whether this tool fits the current need or what a bare parameter like `x` is supposed to hold; writing tool specs the way one would explain the tool to a new colleague, with clear purpose, expected use, and unambiguously named parameters, is documented to produce measurable accuracy gains -- Anthropic reports this kind of refinement alone reducing error rate and raising completion rate on a coding benchmark, with no change to the underlying model. The option claiming names and descriptions are purely cosmetic and ignored is wrong and is the exact misconception this scenario disproves, since the only change made was to that metadata and it moved the measured accuracy. The option claiming the fix is capping the tool count at one is wrong: the scenario kept the same single tool throughout and only rewrote its metadata, so tool count was never the variable in play. The option claiming a rename changes the underlying data type is wrong: renaming `x` to `customer_email` is a documentation and naming change, not a change to what data type the parameter actually accepts.

Source: Anthropic, 'Writing effective tools for AI agents -- using AI agents' (anthropic.com/engineering/writing-tools-for-agents) -- describes optimizing tool descriptions/specs (as if explaining to a new hire, with unambiguous parameter names) measurably reducing error rate and increasing completion rate on SWE-bench, with no model change

Correct answer: .

This is routing: classifying an input and directing it to a specialized downstream prompt built for that category, rather than handling every case with one generalized prompt. It fits here because billing, technical, and general inquiries are distinct, reliably classifiable categories that each benefit from separation of concerns -- a prompt tuned for billing questions can be narrower and more accurate than one that also has to handle technical troubleshooting. The option describing a fixed sequence of subtasks that always runs in the same order regardless of category is wrong because nothing here chains multiple sequential steps; a single classification step chooses one path, it doesn't sequence several. The option describing a second call critiquing and revising the first call's output is wrong because no critique-and-revise loop exists here -- the classification's output is acted on directly, not evaluated and iterated on afterward. The option describing a lead call dynamically spinning up a varying number of parallel workers is wrong because this scenario routes to exactly one specialized prompt per message, not to multiple concurrent workers coordinated by a lead call.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes routing as classifying an input and directing it to a specialized followup task, suited to tasks with distinct categories that are better handled separately and where classification can be handled accurately.

Correct answer: .

This is voting, the parallelization variation that runs the same task multiple times to get diverse outputs, applied here so that a vulnerability only counts as flagged when multiple independent attempts agree, which is meant to produce a higher-confidence result than trusting any single review pass. The option describing sectioning is wrong because sectioning divides one task into different independent pieces that are each reviewed once, whereas this scenario runs the identical, undivided task three separate times rather than splitting the diff into parts. The option describing orchestrator-workers is wrong because that pattern has a lead call dynamically decompose a task and assign distinct subtasks to different workers, whereas here every run gets the exact same prompt and the exact same diff, with no decomposition or specialization involved. The option describing evaluator-optimizer is wrong because that pattern is a sequential critique-and-revise loop between a generator and a separate evaluator, producing one progressively refined result, not several independent parallel attempts whose outputs are combined by requiring agreement.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes voting as 'running the same task multiple times to get diverse outputs,' applied 'when multiple perspectives or attempts are needed for higher confidence results,' citing code vulnerability review by several prompts flagging issues found as an example.

Correct answer: .

MCP defines three core primitives that servers can expose: tools are executable functions that AI applications can invoke to perform actions, resources are data sources that provide contextual information, and prompts are reusable templates that help structure interactions with language models. Mapped onto the scenario, `search_wiki` is a tool because the model decides to invoke it to perform an action; handing over a specific page's raw current text without any model-invoked action is a resource, since it is data supplied for context rather than an action performed; and the pre-filled 'summarize this page' template is a prompt, since it structures an interaction rather than doing anything or supplying raw data itself. The three incorrect options are each wrong for the same reason: they keep the correct three names in play but scramble which definition belongs to which term, for example calling the model-invoked action a 'resource' or the raw contextual data a 'tool,' which inverts MCP's actual definitions of what each primitive is for.

Source: Model Context Protocol documentation, 'Architecture overview' (modelcontextprotocol.io/docs/concepts/architecture, protocol version 2026-07-28) -- 'MCP defines three core primitives that servers can expose: Tools: Executable functions that AI applications can invoke to perform actions... Resources: Data sources that provide contextual information to AI applications... Prompts: Reusable templates that help structure interactions with language models.'

Correct answer: .

The `tool` type of `tool_choice`, given a `name`, forces Claude to always use that one particular tool on the request, which is exactly the guarantee needed here: `submit_form` will be called rather than some other tool or a plain-text reply. The option describing `auto` is wrong because `auto` is the default behavior that lets Claude decide on its own whether to call any provided tool or respond directly, which gives no guarantee that `submit_form` specifically -- or any tool at all -- gets called. The option describing `none` is wrong because it does the opposite of what is wanted: it prevents Claude from calling any tool at all, forcing a plain-text response, when the goal here is to force a specific tool call rather than block tool use entirely. The option describing `any` is wrong because, while it does force Claude to call some tool rather than respond in plain text, it leaves the choice of which tool up to Claude among everything provided, which does not guarantee `submit_form` specifically is the one selected if other tools are also defined on the request.

Source: Anthropic, 'Define tools' (platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools#forcing-tool-use) -- 'auto allows Claude to decide whether to call any provided tools or not... any tells Claude that it must use one of the provided tools, but doesn't force a particular tool... tool forces Claude to always use a particular tool... none prevents Claude from using any tools.'

Correct answer: .

Setting `disable_parallel_tool_use: true` restricts Claude to at most one tool call per turn, which stops it from ever bundling `check_inventory` and `charge_customer` together as simultaneous parallel calls in a single response before the inventory result is available to inform the charge decision. It does not, by itself, encode any notion of which tool must come before which across separate turns -- it only limits how many calls can go out together in one turn, so the calling application still needs its own logic (such as only exposing `charge_customer` once a successful inventory check has already happened) to actually enforce the required ordering. The option claiming it guarantees correct ordering across every future turn is wrong because the parameter carries no dependency information between specific named tools; it is a blanket limit on concurrency, not a sequencing rule. The option claiming it blocks `charge_customer` from ever being called is wrong because the parameter only limits how many tools can be requested at once, not which tools are eligible to be called at all. The option claiming it has no effect on tool-calling behavior is wrong because it directly changes how many tool calls Claude can request in a single turn, which is a behavioral effect, not merely a formatting or logging change.

Source: Anthropic, 'Tool use with Claude' (platform.claude.com/docs/en/agents-and-tools/tool-use/overview) -- example request setting `tool_choice: {type: "auto", disable_parallel_tool_use: true}` with the comment 'Ask for at most one tool call per turn.'

Correct answer: .

Strict tool use works by constraining the model's own token sampling to schema-valid outputs, a technique called grammar-constrained sampling, so that Claude's generated `input` strictly follows the tool's `input_schema` from the moment it is produced -- guaranteeing a correctly-typed integer `passengers` value rather than the application having to detect a mismatch after the fact and retry. The option describing after-the-fact string-to-integer conversion on the application side is wrong because the guarantee happens during generation itself, constraining what Claude can output in the first place, not through a separate post-processing conversion step performed by the calling application. The option describing disabling the parameter's type entirely is wrong and describes the opposite effect: strict mode tightens type enforcement so the schema is always honored, it does not loosen or remove type checking. The option describing mandatory human review of every tool call's arguments is wrong because strict tool use is documented as an automated, per-request generation-time guarantee, with no human-in-the-loop step described as part of the mechanism.

Source: Anthropic, 'Strict tool use' (platform.claude.com/docs/en/agents-and-tools/tool-use/strict-tool-use) -- 'Setting strict: true on a tool definition guarantees Claude's tool inputs match your JSON Schema by constraining the model's token sampling to schema-valid outputs (a technique called grammar-constrained sampling).'

Correct answer: .

Reflexion relies on verbal reinforcement: the agent verbally reflects on feedback about its own failure, stores that self-generated reflective text in an episodic memory buffer, and that stored reflection is fed back into the agent's context on the next attempt to induce better decision-making, all without ever touching the underlying model's weights. The option describing a gradient update from a scalar reward is wrong because that describes conventional reinforcement learning acting on model parameters, which is precisely what Reflexion is designed to avoid -- the improvement here comes from language stored and reused in context, not from a parameter update. The option describing fine-tuning a copy of the model on the failed transcript is wrong for the same reason: it bakes the improvement into new weights, whereas Reflexion's whole mechanism is to leave the weights untouched and instead carry improvement forward as reusable text. The option describing random prompt perturbation is wrong because it describes a trial-and-error process disconnected from any actual feedback about why the previous attempt failed, whereas Reflexion's reflection is specifically generated from and responsive to that feedback.

Source: Shinn et al., 'Reflexion: Language Agents with Verbal Reinforcement Learning' (2023), arXiv:2303.11366 -- describes agents that 'verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials,' without updating model weights.

Correct answer: .

This is the evaluator-optimizer pattern: one LLM call generates a response while a separate call evaluates it and provides feedback in a loop, and it is worth its added latency specifically when there are clear evaluation criteria (here, tone, idiom, and nuance) and when iterative refinement driven by that feedback provides measurable value -- the kind of case where articulated feedback visibly improves the result, as with literary translation's need to capture nuance. The option describing routing is wrong because nothing here classifies the request and dispatches it to one specialized prompt; instead the same draft passes repeatedly between a generator and an evaluator. The option describing prompt chaining with no feedback loop is wrong because a fixed, always-the-same sequence of steps would not involve a separate call critiquing the draft and feeding concrete feedback back for revision -- the defining feature here is exactly that feedback loop, which plain sequential chaining lacks. The option describing parallelization by voting is wrong because voting runs multiple independent attempts at the same task simultaneously and combines them by agreement, whereas this scenario runs one draft through repeated sequential rounds of critique and revision rather than several independent attempts compared against each other.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes evaluator-optimizer as 'one LLM call generates a response while another provides evaluation and feedback in a loop,' appropriate 'when we have clear evaluation criteria, and when iterative refinement provides measurable value,' citing literary translation as a use case.

Correct answer: .

This is a guardrails implementation of the parallelization pattern's sectioning variation: the task is split into independent pieces run simultaneously, one instance handling content screening and a separate instance handling the actual response, and Anthropic's guidance notes this tends to perform better than having the same LLM call handle both guardrails and the core response at once. The option claiming a single call is technically incapable of producing more than one type of output is wrong -- nothing prevents one call from attempting both screening and drafting together; the reason to separate them is that doing so tends to perform better, a quality argument, not a hard technical impossibility. The option claiming two calls are always cheaper than one is wrong and backwards: running two simultaneous calls instead of one generally costs more, not less, so cost reduction is not the reason for this design. The option claiming content screening must legally be performed by a human reviewer is wrong and unsupported -- nothing in this guidance requires human review of every message; the screening call described here is itself an LLM call, not a human one.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- describes guardrails as a sectioning application of parallelization: 'one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.'

Correct answer: .

Marking most tool definitions with `defer_loading: true` keeps only a small non-deferred set of tools, plus the tool search tool itself, loaded into context up front; the rest stay out of context until Claude searches the catalog by tool names, descriptions, and argument details, at which point the API expands only the matching tools it finds into full definitions. This directly addresses both symptoms: it cuts the large upfront token cost that a typical multiserver setup otherwise consumes before any real work happens, and it keeps tool-selection accuracy high even as the catalog grows into the hundreds or thousands, since Claude is only ever choosing among the small set of tools its search actually surfaced. The option describing deletion of the unused definitions is wrong because every tool's full definition still has to be sent in the `tools` array on every request regardless of `defer_loading` -- the API needs them server-side to run the search and expand matches -- deferred tools are kept out of context, not out of the request. The option describing forced sequential calling of every deferred tool at session start is wrong because deferred tools load only when Claude's search discovers them as relevant to the current task, not automatically or unconditionally at the start of a conversation. The option describing merging every deferred tool into one shared schema is wrong because each tool keeps its own distinct name, description, and schema throughout; tool search discovers and expands individual matching tools rather than combining them into a single tool.

Source: Anthropic, 'Tool search tool' (platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool) -- 'Instead of loading all tool definitions into the context window up front, Claude searches your tool catalog... and loads only the tools it needs,' noting a typical multiserver setup can consume roughly 55k tokens up front, that tool search 'typically reduces this by over 85 percent,' and that tool-selection accuracy 'degrades once you exceed 30-50 available tools.'

Correct answer: .

Anthropic's guide draws a specific line between 'workflows,' defined as systems where LLMs and tools are orchestrated through predefined code paths, and 'agents,' defined as systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. System B matches the agent definition: a single model decides, turn by turn, which of its available tools to invoke and when the underlying request has been fully resolved, with no sequence of steps fixed in code ahead of time. System A is the workflow: order lookup, return-window check, refund calculation, and email confirmation always happen in that same order regardless of what the model produces at each step, so the code path -- not the model -- controls what happens next. Completing a task reliably end-to-end doesn't make a system an agent under this distinction; a workflow can be just as reliable, and Anthropic explicitly recommends workflows over agents for well-defined tasks where predictability matters more than flexibility. Using an LLM to generate text at some point doesn't make every system an agent either, since that's true of both systems here; the distinguishing factor is who controls the sequence of actions, not whether an LLM is involved somewhere. Nothing in this definition requires hours of unsupervised operation -- the distinction turns on who decides the next step, not on how long the system runs.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- 'Workflows are systems where LLMs and tools are orchestrated through predefined code paths... Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.'

Correct answer: .

This is prompt chaining: the task -- produce an outline, then expand it into a full post -- is decomposed into a fixed sequence of two LLM calls, where the second call's input is the first call's output. The section-count check between them is exactly what Anthropic's guide calls a 'gate': a programmatic checkpoint on an intermediate step that verifies the process is still on track before letting it proceed, and that can halt the pipeline (routing to a human, as here) rather than pass a bad outline forward. This differs from orchestrator-workers, where a central LLM call dynamically decides what subtasks exist and assigns them to worker calls at runtime -- here the two steps and their order are fixed in advance, not decided by a model. It differs from routing, where an input is classified up front and then sent to one of several different specialized prompts chosen based on that classification -- the check here doesn't choose between different downstream prompts, it only validates one fixed downstream step. And it differs from evaluator-optimizer, where a second LLM call specifically critiques the first call's output in natural language and feeds that critique back for revision in a loop -- the section-count check is a simple programmatic rule, not a separate LLM call judging quality and generating feedback for iteration.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- prompt chaining section: 'decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one... programmatic checks ("gate" in the diagram below) on any intermediate steps to ensure that the process is still on track.'

Correct answer: .

Anthropic's parallelization pattern splits into two variations that solve different problems. Sectioning breaks one task into independent subtasks, each done exactly once, and combines the separate outputs into a single final result -- which is what the manual-translation team does: ten chapters, ten independent translation calls, one concatenated document. Voting runs the identical task multiple times to get diverse independent attempts at the same thing, then combines those attempts by aggregating agreement -- which is what the code-review team does: the same pull request, reviewed three separate times, with a bug flagged only when at least two of the three independent reviews agree it exists. The direction of comparison in one option is reversed: it assigns voting to the team dividing one input into distinct pieces and sectioning to the team repeating identical work, which is exactly backwards relative to Anthropic's definitions. Labeling both setups as sectioning ignores that only one of them divides a single task into distinct, non-overlapping pieces; the other repeats one identical task multiple times, which sectioning by definition does not do. And parallelization in this taxonomy specifically refers to separate calls running independently and being combined only after they finish -- it does not require the calls to communicate with each other mid-execution; requiring that would rule out both of these otherwise-textbook examples of the pattern.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- parallelization section: 'Sectioning: Breaking a task into independent subtasks run in parallel' and 'Voting: Running the same task multiple times to get diverse outputs.'

Correct answer: .

BabyAGI's defining structural feature, compared to a design that commits to one full plan generated upfront and then simply executes each of its steps in order, is that it never commits to a fixed plan at all. Each iteration executes only the single task at the top of its list, then uses the objective plus that task's fresh result to generate new candidate tasks, and then reprioritizes the entire list -- so what the agent intends to do next is recomputed after every single step based on what was just learned, rather than being decided once and followed mechanically. A vector database is commonly used in BabyAGI's implementation to store task context, but nothing about holding more than one pending task inherently requires an external database -- an in-memory list would work architecturally too; the database is an implementation choice for retrieval across a long session, not a structural requirement of having multiple tasks. Generating every task in one single LLM call up front is the opposite of BabyAGI's design, which is exactly why it does not resemble a fixed upfront plan -- new tasks are generated incrementally, one loop iteration at a time, based on results that don't exist yet at the start. And the prioritization agent's job is to reorder tasks based on their relevance and dependencies relative to the objective, not merely to timestamp them by creation order -- creation-order-only sequencing would make the separate prioritization step pointless.

Source: Yohei Nakajima, original BabyAGI README (github.com/yoheinakajima/babyagi_archive) -- describing the execution_agent, task_creation_agent, and prioritization_agent functions run in an infinite loop that pulls the first task, executes it, generates new tasks from the objective and the result, then reprioritizes the list.

Correct answer: .

Toolformer's self-supervised filtering criterion is perplexity reduction: for each candidate position and API call the model proposes throughout its training corpus, the method actually executes the call, inserts the call and its result into the text at that point, and then checks whether doing so improves (reduces perplexity on, i.e., raises the model's predicted probability for) the tokens that come after that point, compared to not making the call. Only examples that pass this check are kept as fine-tuning data teaching the model when and how to call each tool -- calls that don't measurably help predict the continuation are discarded, which is what makes the whole pipeline self-supervised rather than dependent on any human-labeled 'correct tool call' examples. There is no human-written reference-answer dataset involved anywhere in this filtering step; the entire point of Toolformer's method is to avoid needing one. Toolformer is also not restricted to a small fixed whitelist of tools -- the original paper applies the same perplexity-filtering method across several different tool types (a calculator, a question-answering system, a search engine, a translation system, and a calendar), and it's the filtering criterion, not a tool whitelist, that decides what gets kept. Nothing in the method involves a separate compiler pass checking syntactic validity; the check that matters is entirely about whether the inserted call-and-result measurably helps the model predict subsequent text, not whether the call is well-formed code.

Source: Schick et al., 'Toolformer: Language Models Can Teach Themselves to Use Tools,' arXiv:2302.04761 (NeurIPS 2023) -- self-supervised filtering of candidate API calls by whether inserting the call and its result reduces perplexity on subsequent tokens.

Correct answer: .

Computer use lets Claude perceive and act on a screen the same way a person would: it takes screenshots of a virtual desktop and issues mouse-move, mouse-click, and keyboard actions based on what those screenshots show, with no reliance on selectors, an API, or a documented file format for the target application. That's precisely why it fits a legacy application with no API and no documented interface -- the team doesn't need to reverse-engineer that application's internals or build and maintain a custom tool for every one of its screens and workflows; Claude instead navigates it visually, the way a human operator already does. It is not a text-only interface -- the screenshots themselves are the primary input the model reasons over, so vision is central to how it works, not eliminated by it. It also doesn't guarantee pixel-perfect, deterministic execution; actions are still driven by the model's own visual interpretation of each screenshot and can misclick or misjudge coordinates, which is why careful prompting and verification loops matter in practice. And computer use doesn't remove the need for sandboxing -- if anything, giving a model direct mouse-and-keyboard control over a real desktop is exactly the kind of capability that calls for running it in an isolated environment rather than directly against production, for the same reasons agent-generated code execution is sandboxed.

Source: Anthropic, 'Computer use tool' (platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) -- Claude takes screenshots and issues mouse and keyboard actions to control a virtual desktop with no selectors required, unlike selector-based automation tools such as Playwright or Selenium.

Correct answer: .

Voyager's growing skill library is what gives it lifelong learning without ever updating the underlying model's weights: each verified, successful skill is stored as reusable executable code along with a description, so competence developed early in a long session -- like crafting a basic tool -- stays permanently available and can be retrieved and reused, or composed into more complex skills, without the agent having to rediscover the same solution from scratch every time a related situation comes up later. This compounding effect is the paper's central point about open-ended, lifelong agent behavior over an extended session. The library does not involve any fine-tuning of the underlying model's weights at all -- Voyager interacts with the model purely through prompting, which is explicitly why the paper describes it as not requiring parameter updates. The skill library also doesn't replace Voyager's automatic curriculum component; the two are separate, complementary parts of the system, with the curriculum proposing what task to attempt next based on the agent's current state and the library supplying reusable code once tasks are attempted. And stored skills aren't treated as permanently correct with no further verification -- Voyager's iterative prompting mechanism specifically incorporates environment feedback and execution errors, including checking whether a skill still works, rather than trusting every stored skill unconditionally forever.

Source: Wang et al., 'Voyager: An Open-Ended Embodied Agent with Large Language Models,' arXiv:2305.16291 -- describing the ever-growing skill library of executable code as one of Voyager's three key components, enabling lifelong learning without fine-tuning the model.

Correct answer: .

The field is `is_error`, an optional boolean on a `tool_result` content block. Setting it to true tells Claude that the named tool's execution failed rather than returning a normal successful result, so Claude reads the accompanying content as an error description -- here, that the weather service is unavailable -- and can incorporate that into its response to the user (for example, explaining the failure and suggesting a retry) or adjust its next action, rather than misinterpreting the error text as if it were an actual weather reading. There's no `retry` field that causes the API to automatically re-execute a tool call server-side; retrying, if desired, is something the calling application's own code decides to do based on the error, not something the API does on its own. There's no `blocked` field that hides a failed call from Claude; hiding the failure would prevent Claude from ever learning the tool didn't work and defeats the purpose of reporting the error at all. And there's no `fatal` field ending the conversation -- a tool error is not defined as an unrecoverable, session-ending event in the API's design; Claude is expected to keep reasoning and potentially retry or take a different approach after seeing an error result, not stop entirely.

Source: Anthropic, 'Handle tool calls' (platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls) -- 'Handling errors with is_error': 'you can return the error message in the content along with "is_error": true. Claude will then incorporate this error into its response to the user.'

Correct answer: .

MCP and A2A solve different layers of the same problem and are meant to be used together rather than as alternatives. MCP is a vertical protocol connecting one agent's host/client application to the tools, databases, and data sources it personally needs -- exactly the relationship each company's own agent has with its own internal systems here. A2A is a horizontal protocol specifically designed for peer-to-peer collaboration between independently built and operated agents: it lets one agent discover another, delegate part of a task to it, and receive results back, all while each side stays opaque to the other -- neither has to expose its internal tools, prompts, model, or memory, which is exactly the constraint this cross-company hand-off requires. Swapping the two roles is backwards: MCP's own specification concerns a host application talking to tool and data servers, not negotiating task delegation between two independently operated agents, which is squarely A2A's purpose. MCP's architecture was not extended to natively perform agent-to-agent task delegation between separate organizations -- that unmet need is what led Google to develop A2A as a distinct, complementary protocol in the first place, now maintained by the Linux Foundation. And neither protocol requires shared models or common ownership; A2A's entire premise is enabling collaboration precisely when agents come from different frameworks, vendors, and organizations.

Source: A2A Protocol documentation (a2a-protocol.org) -- 'an open standard for seamless communication and collaboration between AI agents,' enabling agents to 'discover each other, delegate tasks, and share results' while remaining opaque to one another; contrasted with the Model Context Protocol (modelcontextprotocol.io), which standardizes agent-to-tool and agent-to-data-source connections.

Correct answer: .

LLMCompiler's Function Calling Planner produces a directed acyclic graph (DAG) representing all four calls and the dependencies between them -- an edge from one call to another whenever the second needs the first's result as an input. A separate Task Fetching Unit then dispatches each call to an Executor as soon as its own dependencies (if any) are satisfied, so the two independent lookups run concurrently with each other while each dependent lookup still waits only for the specific upstream result it actually needs, rather than waiting for every other call in the batch to finish first the way a strictly sequential ReAct-style loop would. The point of representing dependencies this way is exactly to shorten overall latency by exploiting whatever real independence exists between calls, which the paper reports as a substantial speedup over ReAct. A linked list forces every call into one single chain regardless of whether it actually depends on the previous one, which reproduces sequential, one-at-a-time execution rather than enabling any concurrency. A hash table mapping tool names to previously cached results addresses repeated identical queries, not the dependency structure of four calls being made for the first time in this request. And a binary search tree ordering calls by expected runtime says nothing about which calls actually depend on which other calls' outputs, so it could not by itself guarantee a dependent call runs only after the specific result it needs is available.

Source: Kim et al., 'An LLM Compiler for Parallel Function Calling,' arXiv:2312.04511 (ICML 2024) -- the Function Calling Planner generates a DAG of tasks with their inter-dependencies, dispatched by the Task Fetching Unit to the Executor for parallel execution of independent calls, reporting latency speedups of up to 3.7x over ReAct.

Correct answer: .

ReAct, introduced by Yao et al. (2022), interleaves reasoning traces ('Thought') with actions and observations in a loop. The Thought step is where the model explicitly reasons about what it has observed, what it still needs, and what action would best advance toward the goal. This explicit reasoning improves task success because the model can catch errors, decompose complex problems, and adjust its strategy based on observations — unlike pure action-based agents that map observations directly to actions without deliberation. The timeout option confuses human-like pacing with cognitive architecture. The classifier option describes a different architecture (a routing model), not ReAct's self-contained reasoning. The logging-only option underestimates the Thought step — it actively shapes the model's next action through chain-of-thought reasoning, not just records it.

Source: Yao et al. (2022), 'ReAct: Synergizing Reasoning and Acting in Language Models', arXiv:2210.03629

Correct answer: .

Giving an LLM unrestricted shell access is one of the highest-risk tool configurations possible because it provides a direct path from prompt injection to real-world harm. A carefully crafted user input or data returned from a web search could instruct the model to execute destructive commands, and an unrestricted shell would comply. Mitigations include sandboxing (running the shell in an isolated environment), allowlisting specific commands, requiring human approval for sensitive operations, and treating all external content as untrusted data rather than instructions. The speed concern is real but minor compared to the security risk. The preference concern is speculative and depends on the model's tool-selection prompt, not an inherent property of shell tools. The token-cost concern is valid but also trivial relative to the potential for arbitrary code execution by an attacker.

Source: OWASP, 'LLM Top 10: Insecure Plugin Design'; Anthropic, 'Building Safe AI Agents', docs.anthropic.com

Correct answer: .

LangGraph represents an agent's control flow as a directed graph of nodes (each a function that reads and updates shared state) connected by edges, including conditional edges that evaluate a function against the current state to decide which node runs next. Unlike a directed acyclic graph, this structure explicitly supports cycles: a conditional edge can route back to an earlier node -- here, back to 'write_code' when 'run_tests' reports failure -- letting the agent retry, re-plan, or refine its output based on what happened at runtime, something a strictly linear ReAct loop (which only ever proceeds to its next Thought/Action/Observation step) cannot express on its own. The option claiming this guarantees a correct answer is wrong -- routing control flow through a graph says nothing about whether the underlying model reasons correctly at any given node; it only changes what structure the calling code can express. The option claiming nodes run in total isolation with no shared state is wrong and backwards -- LangGraph nodes explicitly read from and write to a common state object that flows through the graph, which is how a later node can see what an earlier one produced. The option requiring every node to call an external tool is wrong -- a node is simply a function that can call an LLM, call a tool, transform data, or do any combination, and plenty of useful nodes (such as a planning step) never call a tool at all.

Source: LangChain, LangGraph 'Graph API' documentation, https://docs.langchain.com/oss/python/langgraph/graph-api

Correct answer: .

OpenAI's Chat Completions API allows a single assistant message to request multiple tool calls at once by populating its `tool_calls` array with more than one entry, each carrying its own unique `id`. The calling application must execute every requested call and then send back one tool-role message per call, with each message's `tool_call_id` set to the matching entry's `id` -- this is how the API pairs each result with the specific request it answers, since the model made two distinct, independently identified requests rather than one. The option combining both results into a single tool message is wrong: the API expects one tool message per tool call, individually tagged by `tool_call_id`, not a single merged reply. The option discarding one of the two calls is wrong and misreads the scenario -- two entries in `tool_calls` means the model deliberately requested two independent lookups in the same turn (the two tools have nothing to do with each other), not that it made an error needing correction. The option waiting for the model to ask for each result one at a time is wrong -- the model has already requested both calls up front in a single turn; there is no further prompt to wait for before executing them.

Source: OpenAI, 'Function calling' API guide, https://developers.openai.com/api/docs/guides/function-calling

Correct answer: .

Always sending every request to the most capable model wastes money on requests a much cheaper model could have handled just as well -- simple classification, structured extraction, and short summarization are typically fine on small models, while deep multi-step reasoning benefits from a stronger one. A learned router, such as the approach described in RouteLLM, is trained on preference data to predict which requests actually need the expensive model and sends only those onward, reported to cut costs substantially while keeping response quality close to always using the strong model. The main risk of this approach is a misrouting: if the router underestimates a request's difficulty and sends a genuinely hard request to the cheap model, the returned answer can be lower quality or outright wrong, since the cheap model was never capable of handling it well in the first place. The option about output length limits is wrong -- routing to a different model changes which model answers, not any hard cap on response length, and token limits are a separate, per-request parameter. The option claiming routing eliminates hallucination and that latency is the only risk is wrong -- switching models does not guarantee factual correctness, and the quality risk of misrouting to a weaker model is exactly what makes routing decisions consequential. The option about training-data staleness is wrong -- the router's job is choosing between models based on predicted task difficulty and cost, not comparing how recently each model was fine-tuned.

Source: Ong et al., 'RouteLLM: Learning to Route LLMs with Preference Data' (2024), arXiv:2406.18665

Correct answer: .

OpenTelemetry's GenAI semantic conventions define a standard schema of spans and attributes -- an 'invoke_agent' span as the top-level unit of work, with child 'chat' spans for each LLM call (capturing attributes like the requested model, input/output token counts, and finish reason) and child 'execute_tool' spans for each tool invocation (capturing the tool name, arguments, and latency) -- so that every step of one specific agent run is captured as a structured, ordered, timestamped record. This is built for debugging and root-causing: when a run misbehaves, an engineer can inspect the full span tree to see exactly which calls happened, in what order, with what inputs and outputs, rather than only knowing whether the final answer happened to be right or wrong. The option describing automatic prompt rewriting is wrong -- tracing only records what happened; it doesn't modify the agent's prompts or configuration on its own. The option claiming a span tree proves the final answer is correct is wrong -- a trace shows what the agent did, not whether what it did produced a correct result; that judgment still requires separate evaluation. The option about reducing token consumption is wrong -- recording spans is an observability side-channel that captures data about a run, it does not compress or alter what gets sent to the model in the conversation itself.

Source: OpenTelemetry, 'Inside the LLM Call: GenAI Observability with OpenTelemetry', https://opentelemetry.io/blog/2026/genai-observability/

Correct answer: .

SWE-bench is built from real GitHub issue-pull request pairs, filtered to those where the merged pull request resolved a stated issue and included test changes, and then validated by running each task's tests before and after the reference fix. Scoring an agent's own patch reuses that same test-based check: the patch must apply cleanly to the repository, must make the 'fail-to-pass' tests -- the tests the original fix added or modified specifically to cover the issue -- pass when they previously failed, and must not break the 'pass-to-pass' tests -- the pre-existing tests that were already passing and should keep passing. This gives an objective, automatically checkable pass/fail signal per task rather than relying on any human or model's subjective read of code quality. The option requiring a character-for-character match to the original developer's patch is wrong -- SWE-bench is explicitly test-based precisely so that a differently-written patch achieving the same passing behavior still counts as resolved. The option describing an LLM judge grading the patch's plausibility is wrong -- the benchmark's headline scoring mechanism is running the actual test suite, not soliciting a model's subjective opinion. The option about rewarding shorter patches is wrong -- nothing in SWE-bench's scoring counts or penalizes lines of code; only whether the specified tests pass or fail determines the result.

Source: Jimenez et al., 'SWE-bench: Can Language Models Resolve Real-World GitHub Issues?' (ICLR 2024), arXiv:2310.06770

Correct answer: .

Multiagent debate, as introduced by Du et al., has multiple instances of a model generate an initial answer with reasoning, then repeatedly exchanges each instance's answer and reasoning with the others so every instance can read what the others produced and revise its own answer across several rounds before a final answer is taken; the authors report this cross-examination reduces hallucinations, since instances often catch and remove each other's mistaken claims. This is fundamentally different from independent voting (a parallelization variation where the same task is run multiple times with no communication between runs, and the final answer is simply whichever response the most independent attempts happened to agree on): voting attempts never see each other's output at all, while debate instances specifically read and respond to one another over multiple rounds. The option claiming the only difference is cost is wrong -- the behavioral mechanism itself differs, since debate instances can change their answer in response to what they read from others, which voting's isolated attempts structurally cannot do. The option requiring different vendors' models is wrong -- the debate method as described uses multiple instances of the same model; nothing about the technique requires mixing vendors. The option claiming debate eliminates any final answer selection is wrong -- the rounds of exchange are meant to converge opinions, but a final answer still has to be read off from the instances' responses at the end, whether or not they reach full agreement.

Source: Du et al., 'Improving Factuality and Reasoning in Language Models through Multiagent Debate' (2023), arXiv:2305.14325

Correct answer: .

AutoGen's GroupChatManager supports several speaker-selection methods -- 'auto', 'manual', 'random', and 'round_robin' -- and 'auto' is the default. In 'auto' mode, after each message the manager sends a select-speaker prompt describing the group's agents and the conversation so far to an LLM, and that LLM's response determines which agent speaks next, letting the choice adapt to what has actually been said rather than following a fixed sequence. This is what lets a group chat behave more like a dynamic conversation than a scripted turn order. The option describing a fixed, predetermined order is wrong -- that describes the separate 'round_robin' method, not 'auto', which is explicitly content-sensitive rather than fixed. The option describing pure random selection with no reference to content is wrong -- that describes the separate 'random' method; 'auto' specifically consults an LLM using the conversation's actual content, not a random draw. The option requiring a human to manually choose every speaker is wrong -- that describes the separate 'manual' method; 'auto' is fully automated and does not require a person to type in a choice after each turn.

Source: Microsoft AutoGen (AG2) documentation, 'GroupChat' API reference, https://docs.ag2.ai/latest/docs/api-reference/autogen/GroupChat/

Correct answer: .

PAL keeps the model responsible for reading a natural-language problem and decomposing it into runnable steps expressed as a program, but offloads actually carrying out those steps to a Python interpreter instead of asking the model to compute the result itself in natural-language tokens. This matters because chain-of-thought prompting, even when its reasoning steps are conceptually correct, is prone to arithmetic and symbolic-manipulation slips when the model performs multi-step calculation token by token; handing that execution to an interpreter that computes exactly means the final result is only as good as the generated program's logic, not also at the mercy of token-level arithmetic errors, and Gao et al. report this lifting few-shot accuracy on benchmarks like GSM8K well above chain-of-thought with the same underlying model. The option claiming the interpreter invents the step-by-step solution from raw text alone is wrong -- decomposing the problem into steps is still the model's job; the interpreter only executes the program the model wrote. The option claiming PAL requires full fine-tuning with no few-shot examples is wrong -- PAL is demonstrated with few-shot prompted code generation, not a requirement for weight updates. The option claiming PAL skips intermediate reasoning and produces only a final numeric token is wrong -- PAL's entire mechanism is generating an explicit program as the intermediate reasoning artifact, which is exactly what gets executed.

Source: Gao et al., 'PAL: Program-aided Language Models' (ICML 2023), arXiv:2211.10435

Correct answer: .

The Agent2Agent (A2A) protocol is built for interoperability between independent, opaque agent systems that don't need to expose their internals to each other, and its discovery mechanism is the 'Agent Card': a JSON document describing an agent's identity (name, description, provider), its service endpoint URL, its supported protocol capabilities (such as streaming or push notifications), and the specific skills it offers, conventionally published at a well-known location such as `/.well-known/agent.json`. A client agent fetches this card to determine whether a remote agent can handle a given task and how to reach and authenticate with it, without either side needing to expose internal tools, prompts, or memory. The option describing publishing the remote agent's full system prompt and memory is wrong and is exactly the opposite of A2A's design goal -- each agent is meant to stay a black box to the other, exposing only the Agent Card's capability description, not its internal reasoning state. The option describing inferring capabilities from a live capture of past conversations is wrong -- A2A discovery relies on a published, structured capability document, not passive observation of other exchanges. The option describing downloading and re-hosting model weights is wrong -- A2A explicitly lets each agent remain on its own separate stack; nothing about the protocol involves transferring model weights between organizations.

Source: Agent2Agent (A2A) Protocol, official specification and 'Agent Discovery' documentation, https://a2a-protocol.org/latest/topics/agent-discovery/

Correct answer: .

SWE-agent's authors found that giving a model the same raw interfaces built for humans (a general-purpose shell, vim) performs worse than a custom-built agent-computer interface designed around how a language model actually consumes and produces text: commands kept simple and compact, feedback kept concise but substantive, and hard-coded guardrails -- such as automatically running a linter after an edit and rejecting the edit outright if it introduces a syntax error -- that catch a mistake immediately and let the model retry rather than letting a small error compound into a much larger failure later. This is a broader claim than tool naming and description clarity: it's about engineering the entire action space and feedback loop an agent operates in, informed by ideas from human-computer interaction, not just wording an individual tool's schema more precisely. The option claiming raw shell access always outperforms a custom interface is wrong and is the opposite of SWE-agent's finding -- the custom, constrained interface is what measurably improved task success. The option claiming linters exist purely for style and have nothing to do with reliability is wrong -- here the linter is used specifically as a guardrail that blocks a broken edit from being accepted, directly protecting the agent's task success rather than enforcing stylistic taste. The option claiming an agent can only succeed with exactly one editing tool is wrong -- SWE-agent's interface includes multiple purpose-built commands (for navigation, viewing, and editing), and the point of the design is which few, well-designed commands exist and how they behave, not restricting the agent to a single command.

Source: Yang et al., 'SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering' (NeurIPS 2024), arXiv:2405.15793

Correct answer: .

MemGPT's virtual context management gives the language model itself a set of functions -- for example, functions to archive part of its current context to external storage and to search that external storage and pull specific entries back in -- and the model calls these functions and receives control back afterward (via an interrupt-like mechanism) to decide what to do next, exactly as an operating system moves data between fast main memory and slower disk storage while giving programs the illusion of far more memory than physically exists. This is a meaningfully different design from an automatic memory-stream pipeline that runs retrieval on its own fixed schedule without the model choosing when or what to page: here, the agent itself is the one deciding, turn by turn, when its active context is cluttered enough to evict something and when a specific piece of archived information needs to be pulled back in for the current task. The option claiming this removes the need for function calls entirely is backwards -- function calls are the exact mechanism MemGPT relies on to move data between memory tiers; without them there is no paging at all. The option claiming this increases the model's true context length is wrong -- the underlying model's actual context window is unchanged; virtual context management works around that fixed limit by moving data in and out of it, not by expanding it. The option describing a purely sequential re-summarization of the whole history every turn is wrong -- that describes a much cruder compaction strategy, not MemGPT's selective, function-call-driven retrieval of only the specific archived entries judged relevant to the current step.

Source: Packer, Fang, Patil, Lin, Wooders & Gonzalez, 'MemGPT: Towards LLMs as Operating Systems' (2023), arXiv:2310.08560

Correct answer: .

WebArena measures success by functional correctness: each of its 812 long-horizon tasks ships with its own programmatic reward function that checks the resulting state of the environment -- for instance, whether an order genuinely exists with the right items, whether a specific pull request comment was actually posted, or whether a returned factual answer matches -- rather than matching the agent's transcript wording or its literal sequence of actions against one fixed reference. This design lets an agent take any working path (clicking through a different menu order, phrasing a confirmation message differently) and still be scored correct, since what's checked is whether the task was genuinely accomplished, not how it was narrated or exactly which clicks were used. The option describing a separate LLM judge scoring the transcript is wrong -- WebArena's headline evaluation mechanism is the programmatic, state-based reward function, not a model's subjective read of the conversation. The option requiring an exact click-and-keystroke match against one human reference is wrong -- that would penalize any agent that solved the task through a different, equally valid sequence of actions, which functional-correctness checking specifically avoids. The option scoring only on speed is wrong -- WebArena's checks are about whether the environment ends up in the correct state, not how many seconds the session took.

Source: Zhou et al., 'WebArena: A Realistic Web Environment for Building Autonomous Agents' (ICLR 2024), arXiv:2307.13854

Correct answer: .

Pass@1 only checks whether an agent succeeded on some single attempt, which can make a genuinely inconsistent agent look fine if the one attempt sampled happens to succeed. Tau-bench's pass^k instead re-runs the identical task -- same simulated user, same policy, same starting database state -- k independent times and only counts the agent as having passed that task if every one of those k trials individually reaches the correct final database state; if even one of the k runs diverges into a different, non-compliant sequence of tool calls and fails, the task counts against the agent under this metric. This directly targets the reliability problem the team observed: an agent that succeeds 7 times out of 8 identical trials looks strong under plain pass@1 (which would credit any single success) but reveals its inconsistency under pass^k, and the paper reports that even strong function-calling models fall below 25% under pass^8 in the retail domain despite passing individual attempts more often than that. The option about deployment across multiple domains is wrong -- domain coverage is a property of the benchmark's task set, not something pass^k measures about a given task. The option framing this as a speed-of-learning metric is wrong -- the k trials are independent repetitions with no fine-tuning or memory carried between them, so there is no learning across attempts to measure. The option describing trajectory-matching against a human reference is wrong -- tau-bench's core scoring already compares the resulting database state to a goal state rather than the click-by-click or call-by-call path taken, and pass^k only adds repetition-based consistency on top of that same state-based check.

Source: Yao et al., 'tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains' (2024), arXiv:2406.12045

Correct answer: .

Instructing a single tool-calling model to treat fetched content as data rather than commands still requires that same model to correctly resist an injected instruction it has already read, on every single occasion, with no second line of defense if that judgment ever fails. The dual-LLM pattern instead removes the exposure itself: the privileged call, which holds the tools and can take real-world action, only ever sees the user's own original request and a controller-produced summary or symbolic placeholder standing in for the untrusted content -- never the untrusted text itself -- while the quarantined call that does read the raw content has no tool access, so even if it gets manipulated by an injected instruction it cannot act on anything. This closes off the path an injection needs: there is no route by which text hidden in a fetched page can ever reach the one model capable of calling send-message or calendar-editing tools. The option claiming the quarantined model is immune to injection by fine-tuning is wrong -- the design assumes the quarantined call can be compromised and neutralizes the risk through isolation, not through claimed model-level immunity. The option in which the privileged model reads the raw untrusted content anyway to double-check the summary is wrong -- that reintroduces exactly the exposure the pattern is built to eliminate. The option requiring identical tool access for both calls is wrong -- the quarantined call is specifically stripped of tool access; giving it tools would undo the entire security benefit of the split.

Source: Simon Willison, 'The Dual LLM pattern for building AI assistants that can resist prompt injection' (2023), simonwillison.net/2023/Apr/25/dual-llm-pattern/

Correct answer: .

The lethal trifecta names three specific preconditions -- access to private or sensitive data, exposure to untrusted content able to carry hidden instructions, and a channel capable of communicating data outside the system -- and argues that an agent is only seriously exposed to attacker-driven data theft when all three are present in the same agent at once. The mechanism is that an attacker plants instruction-like text inside content the agent will process (an email, a fetched web page, a GitHub issue); because language models follow instructions found anywhere in their input, that hidden instruction can direct the agent to pull the private data it has access to and send it out through whatever external communication ability it holds, such as an API call or an outgoing message. Removing any one leg breaks this chain: without access to private data there is nothing worth stealing, without exposure to untrusted content there is no path for a hidden instruction to arrive, and without any external communication ability stolen data has no way to leave the system even if the other two conditions are met. The option about context window size, tool count, and temperature is wrong -- those affect cost and latency, not this specific exfiltration risk. The option about read/write/browse capabilities is wrong -- it substitutes generic capability categories for the actual triad of private-data access, untrusted-content exposure, and external communication. The option about orchestration, approval, and schemas is wrong -- those are largely mitigations and design choices, not the risk factors the trifecta names.

Source: Simon Willison, 'The lethal trifecta for AI agents' (2025), simonwillison.net/2025/Jun/16/the-lethal-trifecta/

Correct answer: .

Without interleaved thinking, a model's thinking allocation is effectively front-loaded: it reasons once before its first action and then must decide subsequent actions from tool results without producing a further visible reasoning block in that same turn. Interleaved thinking instead lets the thinking allocation span the entire assistant turn, so after a tool call returns a result -- such as which specific tests failed and why -- the model can generate a new thinking block reasoning specifically about that result before deciding what to do next, and this reasoning-then-acting cycle can repeat multiple times across several tool calls within one uninterrupted turn. For the coding agent in the scenario, this means it can reason freshly about the actual failing-test output before choosing which file to edit, rather than having committed to an edit plan before ever seeing that output. The option about parallel tool calls is wrong -- issuing multiple tool calls at once is a separate capability of the tool-use API and is not what interleaved thinking specifically adds. The option about an expanded context window is wrong -- interleaved thinking changes when and how often reasoning blocks are produced within a turn, not the size of the underlying context window. The option requiring reasoning to be shown to the end user is wrong -- thinking blocks are a distinct part of the response from the final answer shown to a user, and interleaving them differently within a turn doesn't change whether or how they're surfaced outside the API response.

Source: Anthropic, 'Extended thinking' documentation (platform.claude.com/docs/en/build-with-claude/extended-thinking) -- describes interleaved thinking as letting Claude think between tool calls, reasoning about a tool result before deciding its next action within a single turn

Correct answer: .

In the OpenAI Agents SDK, a handoff is represented to the model as an ordinary tool it can call -- functionally a `transfer_to_X`-style function -- so the triage agent isn't making some separate, abstract routing decision outside the normal tool-calling flow; it is calling a tool like any other, and the SDK's runtime specially recognizes that particular tool call and responds by swapping in the destination agent as the one that now owns and continues the conversation, with the original triage agent no longer involved. This is why a handoff shows up in execution traces just like any other tool call, and why teams building multi-destination systems register one handoff per possible destination and let the model choose among them the same way it would choose among any other tools. The option describing a permanent change to the model's weights is wrong -- a handoff is a per-conversation runtime action, not a fine-tuning event, and it doesn't specialize the underlying model for future conversations at all. The option describing both agents responding simultaneously is wrong -- a handoff transfers ownership of the conversation to exactly one destination agent, not a parallel arrangement where the original agent stays active alongside it. The option describing a message asking the user to manually restart with a different agent is wrong -- the transfer happens transparently within the same ongoing conversation and doesn't require any user action to trigger or complete.

Source: OpenAI Agents SDK, 'Handoffs' documentation (openai.github.io/openai-agents-python/handoffs/)

Correct answer: .

GAIA's 466 questions are deliberately written to have a single unambiguous factual answer -- often a short piece of text or a number -- reached through real-world tool use such as browsing specific web pages, opening attached files, or performing a calculation across multiple steps. Because each question resolves to one checkable answer, grading can be done automatically by comparing the assistant's final answer against the reference, avoiding the gameability and interpretability problems that come with judging longer free-form responses, whether by a human or by another model reading the response and forming a subjective opinion. This is also central to the benchmark's philosophy: the questions are conceptually simple for a human (hence the roughly 92% human score) precisely because their difficulty comes from needing correct, multi-step tool use rather than from needing exceptional reasoning, and an unambiguous target answer is what makes that difficulty measurable at all. The option describing SWE-bench-style unit tests is wrong -- that mechanism is specific to evaluating code patches against a repository's test suite, not GAIA's general factual questions. The option requiring every question to involve executing Python code is wrong -- GAIA's questions span browsing, file handling, and reasoning broadly, not exclusively code execution, and correctness is judged by the final answer, not by whether some code merely ran without error. The option describing majority vote among frontier models is wrong -- GAIA's scoring compares the assistant's answer against a fixed reference answer, not a panel of models guessing at correctness.

Source: Mialon, Fourrier, Swift, Wolf, LeCun & Scialom, 'GAIA: a benchmark for General AI Assistants' (2023), arXiv:2311.12983

Correct answer: .

MCP currently defines stdio and Streamable HTTP as its two standard transport mechanisms, and the local-versus-remote distinction in the scenario maps directly onto them. stdio has the client launch the server as a child process and exchange newline-delimited JSON-RPC messages over that process's standard input and standard output, which suits a purely local utility since it never opens a network port and stays confined to the machine it runs on -- minimal overhead and no network exposure to secure. Streamable HTTP instead has the server expose a network endpoint supporting POST and GET (optionally streaming responses via Server-Sent Events), which is exactly what a shared server needs so that many separate clients, potentially on different machines, can each reach it over the internet. The option putting HTTP on the local server and stdio on the shared one is wrong and backwards -- a purely local integration has no need for a network transport, and stdio's child-process model doesn't support multiple independent remote clients connecting to one running server. The option using stdio for both is wrong -- Streamable HTTP exists specifically to support remote, multi-client access, which stdio's local child-process model cannot provide. The option claiming stdio was removed from the specification is wrong -- stdio remains a current, standard transport alongside Streamable HTTP; it is unrelated to and unaffected by the separate deprecation of other MCP features such as sampling or roots.

Source: Model Context Protocol specification, 'Transports' (modelcontextprotocol.io/specification/2025-03-26/basic/transports)

Correct answer: .

CodeAct's central argument is that a fixed JSON or text action format is inherently limited and inflexible: it can only name one predefined tool call at a time, so composing the output of two tools together, branching on a condition, or looping over several calls has to be flattened into multiple separate steps, if it's expressible at all. Consolidating the agent's actions into executable Python code removes that restriction directly, because code natively supports calling multiple functions together, using control flow like loops and conditionals, holding intermediate variables between steps, and drawing on existing libraries -- all within a single action that a real interpreter then executes, with its output (including errors) fed back as the next observation the agent can react to. This is a broader claim about the action space itself, distinct from PAL's narrower use of generated code specifically to avoid token-by-token arithmetic mistakes in word problems -- CodeAct's motivation and evaluation (including its API-Bank results) concern the flexibility and composability of actions generally, not arithmetic accuracy specifically. The option naming arithmetic mistakes as CodeAct's sole motivation is wrong -- that describes PAL's narrower focus, not CodeAct's stated action-space argument. The option claiming hallucinated tool names are entirely eliminated by design is wrong -- code can still reference undefined names or call functions incorrectly and produce a runtime error; executable code doesn't guarantee every reference resolves correctly, it just makes the resulting error observable and actionable. The option claiming the language model runs the code directly inside its own weights is wrong -- CodeAct still requires a real Python interpreter external to the model to execute the generated code and return its output.

Source: Wang, Chen, Yuan, Zhang, Li, Peng & Ji, 'Executable Code Actions Elicit Better LLM Agents' (ICML 2024), arXiv:2402.01030

Correct answer: .

This is the orchestrator-workers workflow: a central orchestrator LLM call dynamically breaks a task down into subtasks and decides how many there are and what each one involves at runtime, based on the specific input it's given, rather than following a subtask list fixed in advance; it then delegates each subtask to its own worker LLM call, and finally synthesizes the workers' individual results back into one combined output itself. This fits exactly the coding scenario described, since which files need changing and what each file's change should look like genuinely cannot be known until the orchestrator call has inspected the actual repository and request -- precisely the kind of task Anthropic's guide gives as the pattern's motivating example. This is what structurally distinguishes it from parallelization's sectioning variation: sectioning also runs multiple calls on different pieces of one task simultaneously, but sectioning's pieces and their number are fixed by the pipeline's design before anything runs, whereas orchestrator-workers decides that decomposition dynamically, call by call, based on what the input actually requires. The option describing routing is wrong because routing classifies an input up front into one of several predefined categories, each handled by a different specialized prompt chosen in advance -- it never dynamically decomposes one task into an orchestrator-determined number of worker subtasks the way this scenario does. The option describing parallelization's sectioning variation is wrong for the reason just given: its defining feature is that the subtasks are predetermined and fixed, not decided at runtime by a central call inspecting the specific input. The option describing evaluator-optimizer is wrong because that pattern is a sequential critique-and-revise loop between a generator and a separate evaluator producing one progressively refined result, not a one-shot dynamic decomposition-and-synthesis step with no iterative critique involved.

Source: Anthropic, 'Building Effective Agents' (anthropic.com/engineering/building-effective-agents) -- orchestrator-workers section: 'a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results,' contrasted with parallelization's fixed, pre-divided subtasks.

Correct answer: .

Gorilla's retrieval-aware training appends each training example's relevant API documentation directly into the training prompt itself, which the authors describe as teaching the model to use the second half of a prompt (the supplied documentation) to answer the first half (the user's request), rather than depending only on whatever API details got baked into its parameters during fine-tuning. Because the model is trained to condition its generated call on documentation text supplied in the prompt, pairing it with a document retriever at inference lets the system hand it current, possibly updated or version-changed documentation, and the model's output tracks that change without any retraining being needed -- the paper reports this substantially reduces hallucinated calls compared to a version of the model fine-tuned without retrieval-aware training. The option describing it as only a benchmark-construction convenience is wrong because retrieval-aware training is specifically a fine-tuning method that changes how the model learns to use prompt content, not merely a step in assembling the paper's APIBench evaluation set. The option claiming it eliminates hallucination entirely and that retrieval never hurts performance is wrong because the authors explicitly report the opposite nuance: adding retrieval does not always improve results and can sometimes hurt performance, so the paper never claims a guarantee of perfect accuracy. The option claiming it replaces the need for a retriever at inference is backwards -- retrieval-aware training is specifically designed to be paired with a retriever at inference, not to substitute for one; the whole point is adapting to documentation the model never saw memorized during training.

Source: Patil, Zhang, Wang & Gonzalez, 'Gorilla: Large Language Model Connected with Massive APIs' (arXiv:2305.15334, NeurIPS 2024)

Correct answer: .

CAMEL calls this technique inception prompting: both role-playing agents are given a carefully designed initial system prompt pair that lays out the shared task and each agent's specific persona and responsibilities before the autonomous, turn-by-turn conversation begins. This initial steering keeps the two independently reasoning agents focused on cooperatively completing the stated task instead of drifting into generic chit-chat, refusing to continue, or falling into repetitive loops, which the paper identifies as common failure modes of unguided role-play. The framework's primary purpose is to generate large volumes of naturally cooperative multi-agent conversational data autonomously, since once inception prompting sets the two agents on track, the conversation proceeds without a human steering or rating each individual turn. The option describing reward modeling is wrong because it requires a human rating every turn live, which is exactly the ongoing human intervention CAMEL's inception prompting is designed to avoid after the initial setup. The option describing scripted dialogue is wrong because CAMEL's agents generate their own turns autonomously based on their assigned roles and the conversation so far; a fully pre-written fixed exchange with no model-generated content would not be a model-generated conversation at all, and would defeat the purpose of studying autonomous cooperative agent behavior. The option describing self-dialogue is wrong because CAMEL's defining feature is two separate, independently reasoning agent instances genuinely exchanging turns with each other, not a single model instance producing both sides of a conversation by itself.

Source: Li, Hammoud, Itani, Khizbullin & Ghanem, 'CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society' (arXiv:2303.17760, NeurIPS 2023)

Correct answer: .

This describes Mixture-of-Agents (MoA): the architecture assigns models to two distinct roles across its layered structure. Within each layer, every model acts as a 'proposer,' independently reading the original question together with the complete set of answers the previous layer produced, and generating its own continuation that becomes part of the reference material the next layer's proposers draw on. After the configured number of layers, one final, separately designated model acts as the 'aggregator': it reads only the last layer's full set of proposer answers and synthesizes them into the single response returned to the user, rather than generating an independent answer from scratch the way a proposer does. The paper reports this layered proposer-then-aggregator structure lets an ensemble of open-weight models collectively outperform even strong single models on instruction-following benchmarks. The option describing Mixture-of-Experts is wrong because MoE is an architecture internal to one model's own weights, where a learned router sends each token to a small subset of that single model's internal expert sub-networks during a single forward pass -- there are no separate models exchanging natural-language answers across layers, unlike the multi-model, multi-layer setup described here. The option describing self-consistency voting is wrong because voting runs the identical task independently multiple times with no layer ever seeing another layer's output, and simply takes the most common answer, whereas this architecture's defining feature is that each layer explicitly reads the previous layer's full set of answers as reference material. The option describing model merging is wrong because the models here never have their weights combined into one set of parameters; the layers communicate entirely through natural-language answers passed from one layer to the next, with every model's own weights staying completely separate throughout.

Source: Wang et al., 'Mixture-of-Agents Enhances Large Language Model Capabilities' (arXiv:2406.04692, 2024)

Correct answer: .

This is API-Bank, which deliberately splits its evaluation into three levels of increasing difficulty: Call, where the correct API is already identified and the model only needs to produce a correctly parameterized invocation of it; Retrieve+Call, where the model must first identify the right API out of a large pool of real, executable candidate APIs before calling it; and Plan+Retrieve+Call, where the model must autonomously plan and chain together multiple API searches and calls across several steps to satisfy one complex request. Separating these levels matters because it isolates which specific sub-skill is driving a model's success or failure -- correctly filling in a call's parameters, correctly retrieving and selecting the right API among many candidates, or correctly planning a multi-step sequence of calls -- rather than reporting one blended score that could hide a model which, say, reliably calls an API correctly once it's been told which one to use, while actually failing specifically at retrieving the right API or planning multi-step sequences. The option describing SWE-bench is wrong because that benchmark scores a generated code patch against real GitHub issues purely by whether it makes the repository's associated tests pass, with no levels separating API-call parameterization, retrieval, or planning as distinct measured sub-skills. The option describing WebArena is wrong because it scores agents by the resulting functional state of self-hosted websites after a browsing task, never isolating a model's API-call-parameterization accuracy as its own measured dimension. The option describing GAIA is wrong because GAIA reports one score per question verified against a single ground-truth answer across its 466 questions, with no structural split separating API selection from call parameterization or from multi-step planning the way API-Bank's three levels do.

Source: Li et al., 'API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs' (arXiv:2304.08244, EMNLP 2023)