Implementation:Lm sys FastChat Basic Stats

Knowledge Sources	Lm_sys_FastChat
Domains	Data_Processing, Model_Evaluation
Last Updated	2026-02-07 06:00 GMT

Overview

Computes basic usage statistics from FastChat conversation log files, including model call counts, vote tallies, and anonymous battle summaries.

Description

Basic Stats is a monitoring utility within FastChat that aggregates and reports fundamental usage metrics from conversation log files. It processes JSON-formatted log entries to extract information about model usage frequency, user voting patterns, and anonymous battle outcomes. The module supports parallel loading of multiple log files to handle large-scale deployments efficiently.

The core workflow involves loading raw log files (optionally in parallel via a process pool), converting them into a unified pandas DataFrame, and then computing statistics such as per-model conversation counts, vote distributions (including ties and abstentions), and anonymous vote breakdowns. The get_anony_vote_df function specifically filters for anonymous battles where model identities are hidden from users, providing unbiased preference data.

This module serves as the foundation for understanding how the FastChat arena is being used and is typically invoked as part of a larger monitoring pipeline or directly via command-line execution to generate periodic reports.

Usage

Use this module when you need to generate summary reports of FastChat arena activity. It is particularly useful for periodic health checks on arena usage, verifying that models are receiving balanced traffic, and extracting anonymous vote data for downstream Elo rating calculations.

Code Reference

Source Location

• Repository: Lm_sys_FastChat
• File: fastchat/serve/monitor/basic_stats.py
• Lines: 1-220

Signature

def report_basic_stats(log_files: list) -> None:
    """Generate and print basic usage statistics from a list of log files."""

def load_log_files(filename: str) -> list:
    """Load and parse a single JSON log file, returning a list of records."""

def load_log_files_parallel(log_files: list) -> pd.DataFrame:
    """Load multiple log files in parallel using multiprocessing and return a combined DataFrame."""

def get_anony_vote_df(df: pd.DataFrame) -> pd.DataFrame:
    """Filter a DataFrame to include only anonymous battle votes."""

Import

from fastchat.serve.monitor.basic_stats import report_basic_stats

I/O Contract

Inputs

Name	Type	Required	Description
log_files	list[str]	Yes	List of file paths to JSON-formatted FastChat conversation logs
filename	str	Yes	Path to a single log file (used by load_log_files)
df	pd.DataFrame	Yes	DataFrame of conversation records (used by get_anony_vote_df)

Outputs

Name	Type	Description
report	None	report_basic_stats prints statistics to stdout
records	list[dict]	load_log_files returns parsed log entries as a list of dictionaries
df	pd.DataFrame	load_log_files_parallel returns a combined DataFrame of all log records
anony_df	pd.DataFrame	get_anony_vote_df returns a filtered DataFrame containing only anonymous votes

Usage Examples

from fastchat.serve.monitor.basic_stats import (
    report_basic_stats,
    load_log_files_parallel,
    get_anony_vote_df,
)

# Generate a full basic stats report from log files
log_files = ["logs/2024-01-01.json", "logs/2024-01-02.json"]
report_basic_stats(log_files)

# Load logs and extract anonymous votes for Elo computation
df = load_log_files_parallel(log_files)
anony_votes = get_anony_vote_df(df)
print(f"Total anonymous battles: {len(anony_votes)}")

Related Pages

• Principle:Lm_sys_FastChat_Arena_Usage_Statistics
• Implements: Principle:Lm_sys_FastChat_Arena_Usage_Statistics
• Lm_sys_FastChat_Clean_Chat_Data - Cleaning and deduplication of chat log data
• Lm_sys_FastChat_Deduplication - High-frequency prompt deduplication
• Lm_sys_FastChat_Monitor_Markdown - Markdown generation for leaderboard display