Anthropic's prompt caching documentation describes marking a "cache breakpoint" with the `cache_control` parameter. For a request that mixes stable content (tool definitions, a system prompt, a large reference document) with content that changes on every call (the current user turn), where should that breakpoint be placed, and why?
- On the very first token of the entire request, including the tool definitions, so that almost nothing in the request ends up covered by the cached prefix
- On the last content block whose prefix is identical across requests -- that is, after the stable tool definitions, system prompt, and document, and before the changing per-request content -- because the cache only stores what comes before the breakpoint, and placing it on content that changes every request would make the cached prefix's hash change each time, producing no cache hits
- On the changing user message itself, because caching is described as useful specifically for content that differs on every single call
- Nowhere -- Anthropic's API caches every request identically by default with no `cache_control` parameter or configuration needed
Why B? And why not the others?
Correct answer: B. On the last content block whose prefix is identical across requests -- that is, after the stable tool definitions, system prompt, and document, and before the changing per-request content -- because the cache only stores what comes before the breakpoint, and placing it on content that changes every request would make the cached prefix's hash change each time, producing no cache hits
Anthropic's documentation explains that a cache breakpoint should sit on the last block whose prefix is identical across requests, with stable content ordered first (tool definitions, then the system prompt, then long reference documents or few-shot examples) and dynamic per-request content, such as the current user turn, placed after the breakpoint; the API caches writes only up to that point and reads by looking backward for a matching cached prefix, so a breakpoint on ever-changing content would produce a different hash on every call and never hit the cache. The second option is wrong because placing the breakpoint at the very first token would leave the large stable blocks that follow it, such as the tool definitions and system prompt, uncached rather than covered, which defeats the purpose of caching the expensive static prefix. The third option is wrong because it inverts the guidance: caching is valuable specifically for content that repeats unchanged across calls, not for content that differs every time. The fourth option is wrong because the documentation requires explicitly adding a `cache_control` field, whether as a single top-level marker or on specific content blocks, for caching to take effect at all.
Source: Anthropic, "Prompt caching" documentation, platform.claude.com build-with-claude/prompt-caching