Implementation:Lm sys FastChat Remote Logger

Knowledge Sources	Lm_sys_FastChat
Domains	Logging, Async Processing, Monitoring
Last Updated	2026-02-07 06:00 GMT

Overview

Asynchronous HTTP JSON logger with a background daemon thread for sending structured log data to a remote endpoint without blocking the main application.

Description

The remote_logger module implements a lightweight, non-blocking logging system designed for the FastChat serving infrastructure. It uses a singleton pattern via the get_remote_logger() factory function, which checks the REMOTE_LOGGER_URL environment variable on first invocation. If the variable is set, a RemoteLogger instance is created; otherwise, a no-op EmptyLogger is returned. Subsequent calls return the cached global instance.

The RemoteLogger class spawns a background daemon thread at initialization that continuously reads from a queue.Queue. When log(data) is called, the data dictionary is enqueued via put_nowait(), ensuring the caller never blocks on network I/O. The background thread's _send_logs() method processes each queued item by first flattening any nested structures (dictionaries, lists, tuples) into JSON strings for top-level consistency, then sending the data as a JSON payload via requests.post(). Any network errors are caught and logged via Python's standard logging module rather than propagated to the caller.

The EmptyLogger class provides an identical interface with a no-op log() method, allowing calling code to use the logger without conditional checks. This pattern ensures zero overhead when remote logging is not configured, while maintaining a consistent API across all deployment configurations.

Usage

Use this module in FastChat serving components that need to send telemetry, usage statistics, or conversation logs to a centralized collection endpoint. Set the REMOTE_LOGGER_URL environment variable to enable logging. Call get_remote_logger() once at module initialization and use the returned logger's log() method to submit dictionaries of structured data. The daemon thread ensures the main serving loop is never blocked by network requests.

Code Reference

Source Location

• Repository: Lm_sys_FastChat
• File: fastchat/serve/remote_logger.py
• Lines: 1-59

Signature

def get_remote_logger() -> RemoteLogger | EmptyLogger:
    """Factory/singleton returning a RemoteLogger or EmptyLogger based on environment config."""
    ...

class EmptyLogger:
    """No-op logger used when REMOTE_LOGGER_URL is not set."""
    def log(self, _data: dict) -> None: ...

class RemoteLogger:
    """Async HTTP JSON logger with background daemon thread."""
    def __init__(self, url: str) -> None: ...
    def log(self, data: dict) -> None: ...
    def _send_logs(self) -> None: ...

Import

from fastchat.serve.remote_logger import get_remote_logger

I/O Contract

Inputs

Name	Type	Required	Description
REMOTE_LOGGER_URL	str (env var)	No	URL of the remote logging endpoint; when unset, logging is disabled (EmptyLogger is used)
data	dict	Yes (for log())	Dictionary of structured log data; nested dicts/lists/tuples are serialized to JSON strings before sending

Outputs

Name	Type	Description
HTTP POST	JSON payload	Log data sent asynchronously to the configured REMOTE_LOGGER_URL endpoint
None	NoneType	log() returns None; all I/O is handled by the background thread

Usage Examples

# Basic usage in a FastChat serving component
import os
os.environ["REMOTE_LOGGER_URL"] = "https://logs.example.com/api/v1/ingest"

from fastchat.serve.remote_logger import get_remote_logger

logger = get_remote_logger()

# Log a simple event (non-blocking)
logger.log({
    "event": "inference_request",
    "model": "vicuna-7b-v1.5",
    "tokens_generated": 256,
    "latency_ms": 1450,
})

# Log with nested data (nested structures are auto-serialized to JSON strings)
logger.log({
    "event": "conversation_turn",
    "user_message": "Hello!",
    "metadata": {"session_id": "abc123", "turn": 3},  # becomes JSON string
})

# When REMOTE_LOGGER_URL is not set, get_remote_logger() returns EmptyLogger
# and log() calls are no-ops with zero overhead

Related Pages

• Principle:Lm_sys_FastChat_Remote_Event_Logging
• Implements: Principle:Lm_sys_FastChat_Remote_Event_Logging
• Lm_sys_FastChat_Huggingface_API_Inference - Inference pipeline in the same serve module
• Lm_sys_FastChat_Vision_Image - Another serving utility that may produce logs