A team gives an agent a tool named `f1` with the description 'does the thing' and a single parameter named `x`. The agent frequently calls the wrong tool or fills `x` with the wrong kind of value. After renaming the tool to `search_customer_orders`, rewriting its description to explain exactly what it searches and when to use it, and renaming the parameter to `customer_email`, the same model's tool-selection and argument accuracy measurably improves, with no change to the model itself. What does this illustrate?
- How a tool is named and described -- clear, specific, unambiguous language covering what it does, when to use it, and what its parameters mean -- is itself a major factor in whether a model selects the right tool and fills its arguments correctly, independent of the underlying model's capability
- That tool names and descriptions are purely cosmetic metadata which models ignore entirely when deciding which tool to call and how to fill its arguments
- That the only way to fix a model calling the wrong tool is to reduce the total number of tools available to at most one
- That renaming a tool's parameters changes the underlying data type the tool accepts, which is why accuracy improved after the parameter was renamed
Why A? And why not the others?
Correct answer: A. How a tool is named and described -- clear, specific, unambiguous language covering what it does, when to use it, and what its parameters mean -- is itself a major factor in whether a model selects the right tool and fills its arguments correctly, independent of the underlying model's capability
Because a model chooses and fills in tool calls based on the names, descriptions, and parameter documentation it is given, vague or ambiguous wording -- a tool named `f1` described only as 'does the thing' -- gives it little to work with when deciding whether this tool fits the current need or what a bare parameter like `x` is supposed to hold; writing tool specs the way one would explain the tool to a new colleague, with clear purpose, expected use, and unambiguously named parameters, is documented to produce measurable accuracy gains -- Anthropic reports this kind of refinement alone reducing error rate and raising completion rate on a coding benchmark, with no change to the underlying model. The option claiming names and descriptions are purely cosmetic and ignored is wrong and is the exact misconception this scenario disproves, since the only change made was to that metadata and it moved the measured accuracy. The option claiming the fix is capping the tool count at one is wrong: the scenario kept the same single tool throughout and only rewrote its metadata, so tool count was never the variable in play. The option claiming a rename changes the underlying data type is wrong: renaming `x` to `customer_email` is a documentation and naming change, not a change to what data type the parameter actually accepts.
Source: Anthropic, 'Writing effective tools for AI agents -- using AI agents' (anthropic.com/engineering/writing-tools-for-agents) -- describes optimizing tool descriptions/specs (as if explaining to a new hire, with unambiguous parameter names) measurably reducing error rate and increasing completion rate on SWE-bench, with no model change