According to Anthropic's current prompt engineering documentation for Claude, roughly how many examples should a multishot (few-shot) prompt include for best results, and what qualities should those examples have?
- Exactly 1 example is optimal, and using more than 1 measurably degrades Claude's performance on every task
- At least 50 examples are required before Claude can reliably follow the demonstrated pattern at all
- About 3 to 5 examples, each relevant to the actual use case and diverse enough (including edge cases) that Claude does not pick up unintended patterns from the examples
- The examples must all be drawn from the exact same edge case repeated with different wording, since variety between examples confuses Claude about which pattern to follow
Why C? And why not the others?
Correct answer: C. About 3 to 5 examples, each relevant to the actual use case and diverse enough (including edge cases) that Claude does not pick up unintended patterns from the examples
Anthropic's documentation states that including 3 to 5 examples gives the best results for multishot prompting, and that examples should be relevant (mirroring the actual use case closely) and diverse (covering edge cases and varying enough that Claude does not pick up unintended patterns), in addition to being structured with example tags so Claude can distinguish them from instructions. The second option is wrong because the documentation recommends multiple examples specifically, not a single one, and does not claim additional examples degrade performance. The third option is wrong because no 50-example minimum is stated anywhere in the guidance; a small handful is presented as sufficient. The fourth option is wrong because the documentation explicitly recommends diversity across examples, including covering edge cases, rather than repeating one edge case with different wording, which is the opposite of what is recommended and is exactly the kind of narrow pattern that causes Claude to overfit to an unintended regularity.
Source: Anthropic, "Prompting best practices" -- "Use examples effectively," platform.claude.com prompt engineering documentation