Implementation:Huggingface Transformers Compute Basic Statistics

Knowledge Sources	Transformers
Domains	Benchmarking, Performance, Statistics
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete tool for computing descriptive statistics from raw benchmark measurement samples, provided by the HuggingFace Transformers benchmark framework.

Description

compute_basic_statistics is a utility function that takes a list of float measurements and returns a dictionary containing six summary statistics: arithmetic mean (avg), standard deviation (std), minimum (min), median (med), maximum (max), and 95th percentile (p95). It uses NumPy functions for computation. If the input list is empty, all values default to 0. The function is used within BenchmarkResult.pprint to summarize end-to-end latency, time to first token, inter-token latency, and throughput for display. Companion functions add_unit_to_duration and equalize_lengths_and_collate format the statistics with appropriate time units and align them for tabular printing.

Usage

Use compute_basic_statistics whenever you have a list of raw benchmark measurements (latencies, throughputs, etc.) and need to compute summary statistics for reporting, comparison, or regression detection.

Code Reference

Source Location

• Repository: transformers
• File: benchmark_v2/framework/data_classes.py (lines 10-18 for compute_basic_statistics, lines 163-176 for BenchmarkResult.pprint)

Signature

def compute_basic_statistics(measurements: list[float]) -> dict[str, float]:
    return {
        "avg": np.mean(measurements) if measurements else 0,
        "std": np.std(measurements) if measurements else 0,
        "min": np.min(measurements) if measurements else 0,
        "med": np.median(measurements) if measurements else 0,
        "max": np.max(measurements) if measurements else 0,
        "p95": np.percentile(measurements, 95) if measurements else 0,
    }

Import

from benchmark_v2.framework.data_classes import compute_basic_statistics

I/O Contract

Inputs

Name	Type	Required	Description
measurements	list[float]	Yes	A list of raw measurement values (e.g., latencies in seconds, throughputs in tokens/second). May be empty.

Outputs

Name	Type	Description
avg	float	Arithmetic mean of measurements.
std	float	Standard deviation (population) of measurements.
min	float	Minimum value.
med	float	Median (50th percentile) value.
max	float	Maximum value.
p95	float	95th percentile value.

Related Functions

add_unit_to_duration

Converts numeric statistics to human-readable duration strings with appropriate units:

Value Range	Unit	Example
> 3600 seconds	hours	1.50hr
> 60 seconds	minutes	2.33min
> 1 second	seconds	3.45s
> 1 millisecond	milliseconds	45.67ms
> 1 microsecond	microseconds	123.45us
<= 1 microsecond	nanoseconds	789.00ns

BenchmarkResult.pprint

Applies compute_basic_statistics to all metric types and formats the output:

def pprint(self, batch_size: int = 0, num_generated_tokens: int = 0, tabs: int = 0) -> None:
    measurements = {
        "E2E Latency": add_unit_to_duration(compute_basic_statistics(self.e2e_latency)),
        "Time to First Token": add_unit_to_duration(compute_basic_statistics(self.time_to_first_token)),
    }
    if len(self.inter_token_latency) > 0:
        measurements["Inter-Token Latency"] = add_unit_to_duration(
            compute_basic_statistics(self.inter_token_latency)
        )
    if batch_size > 0:
        throughput_stats = compute_basic_statistics(self.get_throughput(batch_size * num_generated_tokens))
        measurements["Throughput"] = {key: f"{value:.2f}tok/s" for key, value in throughput_stats.items()}
    dict_to_pprint = equalize_lengths_and_collate(measurements)
    pretty_print_dict(dict_to_pprint, tabs=tabs)

Usage Examples

Basic Usage

from benchmark_v2.framework.data_classes import compute_basic_statistics

# Raw latency measurements from 20 iterations (in seconds)
latencies = [0.45, 0.43, 0.44, 0.46, 0.42, 0.44, 0.43, 0.45, 0.44, 0.43,
             0.44, 0.45, 0.43, 0.44, 0.46, 0.43, 0.44, 0.45, 0.43, 0.44]

stats = compute_basic_statistics(latencies)
print(stats)
# {'avg': 0.44, 'std': 0.0105, 'min': 0.42, 'med': 0.44, 'max': 0.46, 'p95': 0.4575}

With Duration Formatting

from benchmark_v2.framework.data_classes import compute_basic_statistics, add_unit_to_duration

stats = compute_basic_statistics([0.045, 0.043, 0.044])
formatted = add_unit_to_duration(stats)
print(formatted)
# {'avg': '44.00ms', 'std': '0.82ms', 'min': '43.00ms', 'med': '44.00ms', 'max': '45.00ms', 'p95': '44.90ms'}

Empty Input Handling

stats = compute_basic_statistics([])
print(stats)
# {'avg': 0, 'std': 0, 'min': 0, 'med': 0, 'max': 0, 'p95': 0}

Related Pages

Implements Principle

• Principle:Huggingface_Transformers_Benchmark_Statistics