Principle:InternLM Lmdeploy API Client Integration

Knowledge Sources	LMDeploy API Server LMDeploy
Domains	LLM_Serving, Client_SDK
Last Updated	2026-02-07 15:00 GMT

Overview

A client integration pattern for connecting to LMDeploy API servers using either the official OpenAI Python SDK or LMDeploy's native API client.

Description

API Client Integration provides two approaches for consuming LMDeploy's HTTP API:

• OpenAI SDK: The standard openai Python package works directly since the server implements OpenAI's API specification. Simply point the base_url to the LMDeploy server.
• Native APIClient: LMDeploy provides its own Python client with chat_completions_v1() and completions_v1() methods, supporting both streaming and non-streaming modes.

Both approaches support authentication via API keys, streaming responses, and all generation parameters.

Usage

Use the OpenAI SDK when integrating with existing OpenAI-based codebases (minimal code changes). Use the native APIClient for LMDeploy-specific features or when the openai package is not available.

Theoretical Basis

Client integration follows the Adapter Pattern where the LMDeploy server adapts to the OpenAI protocol:

# Abstract client usage
# Option 1: OpenAI SDK (drop-in replacement)
client = OpenAI(base_url="http://server:23333/v1", api_key="key")
response = client.chat.completions.create(model="model", messages=msgs)

# Option 2: Native client
client = APIClient("http://server:23333")
response = client.chat_completions_v1(model="model", messages=msgs)

Related Pages

Implemented By

• Implementation:InternLM_Lmdeploy_APIClient