passdrill
AI & LLM Engineering · Building with LLM APIs · Card 018/024 hard

A developer marks a `cache_control` breakpoint on a large, static system prompt sent with every request to Anthropic's Messages API. According to Anthropic's documentation, how does this affect billing across the first request and later requests that reuse the same cached prefix?

  1. the first request's cache-write tokens are billed at a discount below the base input-token rate, while every later request that hits the same cache is billed at a premium above the base rate, since maintaining a cache costs more than reading fresh input
  2. cache writes and cache hits are always billed at exactly the same per-token rate as ordinary, uncached input tokens, so prompt caching only ever provides a latency benefit and never a cost benefit
  3. the first request that establishes the cache is billed at a premium above the base input-token rate for the cached portion, since writing a new cache entry costs more, while later requests whose prefix hash matches that cached entry are billed at a steep discount for the reused portion, which is where the overall cost savings come from
  4. prompt caching only ever reduces the cost of output tokens, never input tokens, because it works by shortening how much text the model generates in its response rather than by reusing previously processed input
Next card → Shuffle

Keep drilling

Take the full Building with LLM APIs quiz (24 cards)

Report a problem with this question