Principle:Googleapis Python genai Content Generation With Cache
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Generative_AI |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
A generation pattern that references pre-cached context to reduce latency and cost for queries against large, previously stored content.
Description
Content Generation With Cache extends standard generation by referencing a previously created cache instead of re-sending large context with each request. The cache resource name is passed in the generation config, and the model uses the cached context as if it were part of the input. Only the new query content needs to be sent, dramatically reducing input token costs and latency. The model used for generation must match the model specified when creating the cache.
Usage
Use cached generation after creating a cache via caches.create. Pass the cache's resource name in config.cached_content. Send only the new query as contents. This is ideal for question-answering over documents, multi-query analysis, and any scenario where the base context remains constant across requests.
Theoretical Basis
Cached generation separates context processing from query processing:
# Standard: context + query processed together each time
response = model.generate(context + query) # O(C + Q)
# Cached: context pre-processed, only query processed per-call
cache = preprocess(context) # O(C) one-time
response = model.generate(query, cache) # O(Q) per-call
For N queries over context of size C, the savings are (N-1) * C tokens.