Prompt Caching: When It Saves Money—and When It Does Not

The discount is not the complete calculation

A recurring agent prompt may contain instructions, tool definitions and reference documents that barely change. Prompt caching can reuse processing for a shared prefix. That can reduce eligible input processing, but it is not a saved final answer and it does not make output tokens free.

OpenAI and Anthropic document model-specific eligibility, cache-write/read pricing and retention behavior. Those details change. This guide therefore uses a transparent hypothetical example rather than presenting a universal current vendor quote. Check the linked documentation and the selected model before budgeting.

Put stable content before changing content

Caching depends on a matching reusable prefix, subject to the provider’s rules. Keep stable instructions and reference material ahead of the changing question where your API design supports it. Inserting a unique timestamp or request identifier near the beginning can prevent later content from matching.

Do not restructure a prompt blindly. Tool definitions, images, schemas, message order and relevant settings may affect matching. A session that retains context is not automatically a cache hit. Treat usage fields returned by your API as evidence; do not infer successful caching from similar answers.

A worked example with all the assumptions

Assume a reusable 10,000-token prefix, an ordinary input rate of $2 per million tokens, a cache-write rate of $2.50 and a cache-read rate of $0.20. These rates are invented for arithmetic, not a vendor price list. Assume all repeated requests hit the cache within the eligible lifetime.

Without caching, that prefix costs $0.02 each time. The cold write costs $0.025 and each subsequent read costs $0.002. Across ten requests, uncached prefix cost is $0.20; cached prefix cost is $0.025 + 9 × $0.002 = $0.043. Prefix-only savings are $0.157. Dynamic input, generated output, tools, retries and taxes remain outside this comparison.

Find the break-even point

For a prefix of P tokens, let U be its uncached cost per request, W its initial cache-write cost and R each successful read. Compare N × U with W + (N − 1) × R. If U exceeds R, caching is cheaper when N > (W − R) / (U − R). Use complete request costs as well before making a purchasing decision.

In the example, one request costs more with a cache write: $0.025 versus $0.02. Two successful requests cost $0.027 versus $0.04, so the second request crosses break-even. That conclusion depends entirely on those assumptions. A higher write premium, short lifetime or low hit rate can change it.

The four ways a promising estimate fails

• The prefix changes, so a later request cannot reuse the expected entry.
• Requests arrive after expiry or route differently, so apparent repetition does not produce a hit.
• The prefix is below model-specific eligibility thresholds.
• The reusable prefix is a small part of the bill; output, retries or tool calls dominate total spend.

Measure a rollout instead of trusting the spreadsheet

Start with a representative request and record total input, cached input or cache-write/read usage fields where provided, output, latency and provider charges. Compare cold and warm runs without silently discarding misses. Repeat after changing the question and after changing the prefix.

Use a small budget and non-sensitive test data. Confirm applicable retention and data-handling rules before caching private documents. Report the model, API version, lifetime and sample size alongside any result. We verified the worked arithmetic locally; we did not run a paid provider benchmark for this article.

When to skip the optimization

If most prompts are unique, rarely repeated or dominated by generated output, spend time reducing unnecessary requests first. Caching is one part of cost engineering. A shorter reliable workflow can beat a heavily cached loop that keeps retrying an impossible task.

Sources and related reading

Official documentation checked 5 November 2026. Documentation-based research; illustrative scenarios are labeled. Product availability and pricing can change.

OpenAI prompt caching

Anthropic prompt caching

API access and platform trade-offs

Measure eval quality before optimizing