Implementation:LMCache LMCache Usage Context

Knowledge Sources	LMCache
Domains	Usage Tracking, Telemetry
Last Updated	2026-02-09 00:00 GMT

Overview

This module implements usage tracking and telemetry for LMCache, sending environment, engine configuration, and continuous caching statistics to a remote tracking server.

Description

The usage_context.py module provides two main tracking mechanisms. The UsageContext class collects and sends one-time environment information (cloud provider, CPU/GPU details, installation source) and engine configuration at startup. The ContinuousUsageContext singleton periodically aggregates and transmits ongoing caching statistics (stored tokens, hit tokens, cache lifespan histograms) to a configurable remote server. Both classes support local file logging as a fallback. Usage tracking can be disabled via the LMCACHE_TRACK_USAGE=false environment variable.

Usage

The module is automatically invoked during LMCache engine initialization via InitializeUsageContext. The ContinuousUsageContext is created as a singleton and updated by the stats logger thread at regular intervals.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/usage_context.py
• Lines: 1-412

Signature

class EnvMessage:
    def __init__(
        self, provider, num_cpu, cpu_type, cpu_family_model_stepping,
        total_memory, architecture, platforms, gpu_count, gpu_type,
        gpu_memory_per_device, source,
    ): ...

class EngineMessage:
    def __init__(self, config: LMCacheEngineConfig, metadata: LMCacheMetadata): ...

class MetadataMessage:
    def __init__(self, start_time, duration): ...

@dataclass
class ContinuousContextMessage:
    interval_num_stored_tokens: int
    interval_num_hit_tokens: int
    interval_stored_kv_size: int
    message_type: str = "ContinuousContextMessage"

class UsageContext:
    def __init__(
        self, server_url: str, config: LMCacheEngineConfig,
        metadata: LMCacheMetadata, local_log: Optional[str] = None,
    ): ...
    def send_env_message(self): ...
    def send_engine_message(self): ...
    def send_metadata_message(self): ...

class ContinuousUsageContext:
    def __init__(self, metadata: LMCacheMetadata): ...
    @staticmethod
    def GetOrCreate(metadata: LMCacheMetadata) -> "ContinuousUsageContext": ...
    def send_caching_message(self): ...
    def incr_or_send_stats(self, stats: "LMCacheStats"): ...

def InitializeUsageContext(
    config: LMCacheEngineConfig, metadata: LMCacheMetadata,
    local_log: Optional[str] = None,
) -> Optional[UsageContext]: ...

Import

from lmcache.usage_context import (
    UsageContext,
    ContinuousUsageContext,
    InitializeUsageContext,
    EnvMessage,
    EngineMessage,
)

I/O Contract

Inputs

Name	Type	Required	Description
config	LMCacheEngineConfig	Yes	Engine configuration for tracking chunk size, local device, remote URL, etc.
metadata	LMCacheMetadata	Yes	Metadata with model_name, world_size, worker_id, kv_dtype, kv_shape
server_url	str	Yes (for UsageContext)	Remote server URL to send tracking messages to
local_log	str	No	Optional local file path for logging messages
stats	LMCacheStats	Yes (for incr_or_send_stats)	Statistics snapshot from the observability monitor

Outputs

Name	Type	Description
UsageContext	object or None	Initialized usage context, or None if tracking is disabled
HTTP POST requests	side effect	Environment, engine, metadata, and continuous caching messages sent to the tracking server
Local log entries	side effect	Messages written to local log file if configured

Usage Examples

from lmcache.usage_context import InitializeUsageContext, ContinuousUsageContext

# Initialize usage tracking at engine startup
usage_ctx = InitializeUsageContext(config, metadata, local_log="/tmp/lmcache_usage.log")

# Access the continuous usage context singleton
continuous_ctx = ContinuousUsageContext.GetOrCreate(metadata)

# Accumulate stats and send when interval is reached
from lmcache.observability import LMCStatsMonitor
monitor = LMCStatsMonitor.GetOrCreate()
stats = monitor.get_stats_and_clear()
continuous_ctx.incr_or_send_stats(stats)

# Disable tracking via environment variable:
# export LMCACHE_TRACK_USAGE=false