Implementation:InternLM Lmdeploy RequestLogger

Knowledge Sources	InternLM_Lmdeploy
Domains	Logging, Serving, Utilities
Last Updated	2026-02-07 15:00 GMT

Overview

A request logging utility class that logs incoming inference requests with optional truncation, adapted from vLLM's logger module.

Description

The RequestLogger class in lmdeploy/logger.py provides structured logging for inference requests processed by the lmdeploy serving layer. It ensures that log entries do not exceed a configurable maximum length, which is important for avoiding excessively large log files when handling long prompts or base64-encoded image data.

The class provides two logging methods:

• log_prompt: Logs just the session ID and prompt text. Skips logging entirely if the prompt is not a string (e.g., multimodal GPT-4V messages containing base64 images). Truncates the prompt to max_log_len if set.

• log_inputs: Logs detailed request information including session ID, adapter name, input token count, generation configuration, prompt text, and prompt token IDs. Both the prompt text and token IDs are truncated to max_log_len if set.

Both methods use the lmdeploy logger at the INFO level.

Usage

Used by the lmdeploy API server and serving components to log incoming requests for debugging and monitoring purposes.

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: lmdeploy/logger.py
• Lines: 1-50

Signature

class RequestLogger:
    def __init__(self, max_log_len: Optional[int]) -> None: ...

    def log_prompt(self, session_id: int, prompt: str) -> None: ...

    def log_inputs(self, session_id: int, prompt: Optional[str],
                   prompt_token_ids: Optional[List[int]],
                   gen_config: GenerationConfig,
                   adapter_name: str) -> None: ...

Import

from lmdeploy.logger import RequestLogger

I/O Contract

Inputs

Name	Type	Required	Description
max_log_len	Optional[int]	Yes (constructor)	Maximum length of logged strings; None for no limit
session_id	int	Yes	Unique session identifier for the request
prompt	str	Yes (log_prompt)	The prompt text to log
prompt_token_ids	Optional[List[int]]	Yes (log_inputs)	Tokenized prompt IDs
gen_config	GenerationConfig	Yes (log_inputs)	Generation configuration parameters
adapter_name	str	Yes (log_inputs)	Name of the LoRA adapter being used

Outputs

Name	Type	Description
Log entries	side effect	INFO-level log messages written via the lmdeploy logger

Usage Examples

from lmdeploy.logger import RequestLogger
from lmdeploy.messages import GenerationConfig

# Create logger with max 1000 character truncation
request_logger = RequestLogger(max_log_len=1000)

# Log a simple prompt
request_logger.log_prompt(session_id=42, prompt="What is the meaning of life?")

# Log detailed request inputs
request_logger.log_inputs(
    session_id=42,
    prompt="What is the meaning of life?",
    prompt_token_ids=[1, 1724, 338, 278, 6593],
    gen_config=GenerationConfig(max_new_tokens=512, temperature=0.7),
    adapter_name="default"
)

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime