What is prompt caching? Reusing computation for repeated context
Understand prompt caching through repeated manual questions, including prefix matching, lifetimes, usage checks, and differences from answer caching or permanent memory.
Understand the concept
Prompt caching reuses computation state for repeated input. It differs from returning a saved answer; the model still generates for the new question. Hits depend on matching rules, lifetime, model, and settings. It neither expands context capacity nor guarantees free repeated requests or identical answers.
Separate repeated reference material from changing questions
For repeated questions about a manual, keep stable material unchanged and place varying questions afterward. Prefix reuse needs matching content; changing timestamps at the beginning can disrupt reuse. Check provider-specific breakpoint requirements.
OpenAI explains reuse of KV computation while new input is still processed. Google provides implicit caching and differs by model and interface in explicit-cache support. Older instructions for creating cache objects cannot be applied universally to new interfaces.
A cache hit does not establish that content is current
Caching reuses supplied material rather than refreshing an old manual. Send revised content and confirm the request changed. Application-level answer caches need their own invalidation; their update rules differ from prompt-cache rules.
Identical inputs need not produce identical outputs because generation can vary. Inspect model and settings before concluding caching failed. Identical answers also do not prove hits; use reported cached usage.
Account for both cache writes and reuse
Services can price writes and reads differently, with varying minimum lengths and lifetimes. Check cached tokens and total input against current prices. A high hit rate alone does not establish lower overall cost.
Compaction can reduce input while changing prefixes and lowering reuse, so compare totals. This is an accounting explanation without paid requests or fixed savings. For nonrepeated material, removing irrelevant input is easier to justify than padding solely for cache eligibility.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.