Principle:LMCache LMCache CacheBlend Configuration
| Knowledge Sources | |
|---|---|
| Domains | Configuration, Inference_Optimization |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A configuration pattern that enables non-prefix KV cache reuse through segment-based caching and RoPE position recovery parameters.
Description
CacheBlend Configuration extends the base LMCacheEngineConfig with fields that control the CacheBlend algorithm: enable_blending activates the feature, blend_recompute_ratios controls what fraction of positions to recompute (e.g., 0.15 = top 15% most divergent), blend_check_layers specifies which layers to evaluate divergence, blend_thresholds sets divergence thresholds, blend_min_tokens sets the minimum tokens for blending, and blend_special_str defines the separator string between reusable segments.
When blending is enabled, validation auto-sets save_unfull_chunk=True (segments may not align to chunk boundaries) and use_layerwise=True (blending operates per-layer).
Usage
Enable when you need to reuse KV caches from non-prefix text segments (e.g., reordered RAG chunks, shared knowledge bases accessed in different order).
Theoretical Basis
CacheBlend relaxes the prefix constraint of traditional KV caching:
- Traditional: Only exact prefix matches can reuse KV cache
- CacheBlend: Any text segment can be reused regardless of position, with RoPE position recovery correcting the positional encoding mismatch
Configuration controls the accuracy-speed tradeoff via recompute_ratio: higher values recompute more divergent positions (more accurate, slower).