Implementation:LMCache LMCache LMCacheEngineConfig Blend Fields

Knowledge Sources	LMCache
Domains	Configuration, Inference_Optimization
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for configuring CacheBlend parameters provided by the LMCacheEngineConfig class.

Description

The LMCacheEngineConfig includes blend-specific fields in _CONFIG_DEFINITIONS (lines 101-122): enable_blending, blend_recompute_ratios, blend_thresholds, blend_check_layers, blend_min_tokens, and blend_special_str. Validation at lines 501-507 auto-sets save_unfull_chunk=True when blending is enabled.

Usage

Set via environment variables (LMCACHE_ENABLE_BLENDING=True, LMCACHE_BLEND_SPECIAL_STR, etc.) or YAML config file.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/config.py
• Lines: L101-L122 (field definitions), L501-L507 (validation)

Signature

# Blend-specific fields in _CONFIG_DEFINITIONS:
"enable_blending": {"type": bool, "default": False}
"blend_recompute_ratios": {"type": Optional[list[float]], "default": None}
"blend_thresholds": {"type": Optional[list[float]], "default": None}
"blend_check_layers": {"type": list[int], "default": None}
"blend_min_tokens": {"type": int, "default": 256}
"blend_special_str": {"type": str, "default": " # # "}

Import

from lmcache.v1.config import load_engine_config_with_overrides

I/O Contract

Inputs

Name	Type	Required	Description
enable_blending	bool	Yes	Enable CacheBlend mode
blend_recompute_ratios	list[float]	No	Fraction of positions to recompute per layer group (e.g., [0.15])
blend_check_layers	list[int]	No	Layer indices to check for divergence (e.g., [1])
blend_thresholds	list[float]	No	Divergence thresholds
blend_min_tokens	int	No	Minimum tokens for blending (default 256)
blend_special_str	str	No	Separator string between segments (default " # # ")

Outputs

Name	Type	Description
config	LMCacheEngineConfig	Validated config with blending enabled, save_unfull_chunk=True

Usage Examples

Environment Variables

import os
os.environ["LMCACHE_ENABLE_BLENDING"] = "True"
os.environ["LMCACHE_BLEND_SPECIAL_STR"] = " # # "
os.environ["LMCACHE_USE_LAYERWISE"] = "True"
os.environ["LMCACHE_BLEND_CHECK_LAYERS"] = "1"
os.environ["LMCACHE_BLEND_RECOMPUTE_RATIOS"] = "0.15"
os.environ["LMCACHE_CHUNK_SIZE"] = "256"

from lmcache.v1.config import load_engine_config_with_overrides
config = load_engine_config_with_overrides()

YAML Config

# example_blending.yaml
chunk_size: 256
local_cpu: True
max_local_cpu_size: 5
enable_blending: True
blend_special_str: " # # "
use_layerwise: True
blend_check_layers: [1]
blend_recompute_ratios: [0.15]

Related Pages

Implements Principle

• Principle:LMCache_LMCache_CacheBlend_Configuration