Implementation:LMCache LMCache Internal API Server

Knowledge Sources	LMCache
Domains	API Server, Internal Services
Last Updated	2026-02-09 00:00 GMT

Overview

Implements the internal HTTP/Unix socket API server for LMCache, providing runtime introspection and management endpoints for scheduler and worker processes.

Description

The InternalAPIServer class wraps a FastAPI application served by uvicorn, providing internal API endpoints for the LMCache system. At module load time, a global FastAPI app is created and all API routes (common, vllm, controller) are auto-registered via APIRegistry. The server supports both TCP port-based and Unix domain socket-based communication. Port assignment is computed from a configurable base port plus an offset (0 for scheduler, 1+worker_id for workers). The server can be selectively enabled/disabled per process via include_index_list configuration. It runs in a daemon thread and stores a reference to the LMCacheManager on the app state for endpoint handlers to access. The server supports configurable access logging and log levels.

Usage

Use InternalAPIServer to expose internal management APIs from each LMCache process (scheduler or worker). Instantiate with an LMCacheManager, call start() to launch the server in a background thread, and stop() to shut it down. Access endpoints via HTTP requests to the configured port or Unix socket.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/internal_api_server/api_server.py
• Lines: 1-120

Signature

class InternalAPIServer:
    def __init__(self, lmcache_manager: "LMCacheManager") -> None: ...
    async def run(self) -> None: ...
    def start(self) -> None: ...
    def stop(self) -> None: ...

Import

from lmcache.v1.internal_api_server.api_server import InternalAPIServer

I/O Contract

Inputs

Name	Type	Required	Description
lmcache_manager	LMCacheManager	Yes	The LMCache manager providing engine config and metadata
internal_api_server_enabled	bool (config)	No	Whether the server is enabled (from LMCacheEngineConfig)
internal_api_server_port_start	int (config)	No	Base port number for the API server
internal_api_server_socket_path_prefix	str (config)	No	Unix socket path prefix; if set, uses UDS instead of TCP
internal_api_server_host	str (config)	No	Host address to bind the API server to
internal_api_server_include_index_list	list (config)	No	List of port offsets to enable; empty means enable all

Outputs

Name	Type	Description
HTTP API server	(side effect)	Running uvicorn server accessible via TCP port or Unix domain socket

Usage Examples

from lmcache.v1.internal_api_server.api_server import InternalAPIServer

# Create and start the internal API server
api_server = InternalAPIServer(lmcache_manager)
api_server.start()  # Runs in a background daemon thread

# Later, during shutdown
api_server.stop()