Implementation:Microsoft DeepSpeedExamples Plot Loss

Knowledge Sources	Microsoft_DeepSpeedExamples
Domains	Visualization, Training Analysis
Last Updated	2026-02-07 12:00 GMT

Overview

A plotting utility that generates loss curve comparison charts from CSV training logs, with configurable moving-average smoothing for visual clarity.

Description

plot_loss.py is a visualization script that creates side-by-side loss curve comparison plots for baseline (fp32 master weights) versus BF16 low-precision training configurations. It reads loss data from CSV files containing step and loss columns, applies a configurable moving-average smoothing window, and produces a publication-quality PNG chart using matplotlib.

The script renders both raw loss values (with low opacity for context) and smoothed curves (with full opacity and thicker line width) on the same axes. It uses blue for baseline and orange for BF16 curves. A text annotation box in the upper-right corner displays the final loss values for both configurations, making it easy to assess convergence parity.

Additionally, the script computes and prints summary statistics to stdout including final loss, mean loss, and standard deviation for both configurations. The smoothing function uses numpy convolution with front-padding to maintain array length consistency.

Usage

Use this script after training experiments to visually compare loss convergence between baseline and BF16 low-precision master weight configurations. It is ideal for generating figures for reports, papers, and README documentation that demonstrate training quality is preserved with lower-precision master weights.

Code Reference

Source Location

• Repository: Microsoft_DeepSpeedExamples
• File: training/bf16_master_weight/plot_loss.py
• Lines: 1-93

Signature

def load_loss_data(filepath):
    """Load loss data from CSV file."""

def smooth_curve(values, window=10):
    """Apply moving average smoothing."""

def main():
    parser = argparse.ArgumentParser(description="Plot loss curves comparison")
    parser.add_argument("--baseline", type=str, required=True, help="Baseline loss CSV file")
    parser.add_argument("--bf16", type=str, required=True, help="BF16 low-precision loss CSV file")
    parser.add_argument("--output", type=str, default="loss_comparison.png", help="Output plot file")
    parser.add_argument("--smooth", type=int, default=20, help="Smoothing window size")
    parser.add_argument("--title", type=str, default="Loss Comparison: Baseline vs BF16 Low-Precision", help="Plot title")

Import

from training.bf16_master_weight.plot_loss import load_loss_data, smooth_curve, main

I/O Contract

Inputs

Name	Type	Required	Description
--baseline	str	Yes	Path to baseline loss CSV file with 'step' and 'loss' columns
--bf16	str	Yes	Path to BF16 low-precision loss CSV file with 'step' and 'loss' columns
--output	str	No	Output file path for the plot image; defaults to loss_comparison.png
--smooth	int	No	Moving-average smoothing window size; defaults to 20
--title	str	No	Plot title string; defaults to "Loss Comparison: Baseline vs BF16 Low-Precision"

Outputs

Name	Type	Description
plot image	PNG file	Comparison chart saved at 150 DPI with raw and smoothed loss curves
statistics	str (stdout)	Printed final loss, mean, and standard deviation for both configurations

Usage Examples

# Command-line usage
# python plot_loss.py --baseline logs/baseline_loss.csv --bf16 logs/bf16_loss.csv --output loss_comparison.png

# With custom smoothing window
# python plot_loss.py --baseline logs/baseline_loss.csv --bf16 logs/bf16_loss.csv --smooth 50

# Programmatic usage of individual functions
import numpy as np
from training.bf16_master_weight.plot_loss import load_loss_data, smooth_curve

steps, losses = load_loss_data("logs/baseline_loss.csv")
smoothed = smooth_curve(losses, window=20)
print(f"Final loss: {losses[-1]:.4f}, Smoothed final: {smoothed[-1]:.4f}")