Implementation:Microsoft DeepSpeedExamples MoQ Glue Runner

Knowledge Sources	Microsoft_DeepSpeedExamples
Domains	Natural Language Processing, Model Quantization
Last Updated	2026-02-07 12:00 GMT

Overview

GLUE benchmark runner for the MoQ (Mixture of Quantization) example that fine-tunes HuggingFace transformer models for sequence classification tasks with quantization-aware training support.

Description

This module implements a GLUE benchmark fine-tuning script for the MoQ (Mixture of Quantization) training example. It is adapted from the HuggingFace Transformers library's run_glue.py and supports all nine GLUE tasks: CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST-2, STS-B, and WNLI. The script uses HuggingFace's Trainer API with AutoModel classes for flexible model selection and supports both single-sentence and sentence-pair classification tasks.

The module defines two dataclass-based argument classes: DataTrainingArguments for specifying the GLUE task, sequence length, padding behavior, and custom data files; and ModelArguments for configuring the pretrained model path, tokenizer, caching, and model revision. The main() function orchestrates the complete pipeline including argument parsing (from CLI or JSON), checkpoint detection and resumption, logging configuration, dataset loading (from GLUE hub or local CSV/JSON files), tokenization, model initialization, and training/evaluation via the HuggingFace Trainer.

The script supports custom data files (CSV or JSON) for non-GLUE tasks, dynamic or fixed padding strategies, automatic label detection from the dataset, and configurable evaluation metrics based on the selected task (Matthews correlation for CoLA, Pearson/Spearman for STS-B, accuracy and F1 for MRPC/QQP, accuracy for others). It integrates with DeepSpeed through the HuggingFace Trainer's DeepSpeed support for quantization-aware training.

Usage

Use this script to fine-tune pretrained language models on GLUE benchmark tasks as part of the MoQ quantization example. It supports all standard GLUE tasks and can also be adapted for custom text classification tasks by providing training and validation files in CSV or JSON format.

Code Reference

Source Location

• Repository: Microsoft_DeepSpeedExamples
• File: training/MoQ/run_glue.py
• Lines: 1-561

Signature

@dataclass
class DataTrainingArguments:
    task_name: Optional[str] = None
    max_seq_length: int = 128
    overwrite_cache: bool = False
    pad_to_max_length: bool = True
    train_file: Optional[str] = None
    validation_file: Optional[str] = None
    test_file: Optional[str] = None

@dataclass
class ModelArguments:
    model_name_or_path: str
    config_name: Optional[str] = None
    tokenizer_name: Optional[str] = None
    cache_dir: Optional[str] = None
    use_fast_tokenizer: bool = True
    model_revision: str = "main"
    use_auth_token: bool = False

def main():

def _mp_fn(index):

Import

from run_glue import main, DataTrainingArguments, ModelArguments

I/O Contract

Inputs

Name	Type	Required	Description
model_name_or_path	str	Yes	Pretrained model name or path (e.g., "bert-base-uncased")
task_name	str	No	GLUE task name (cola, mnli, mrpc, qnli, qqp, rte, sst2, stsb, wnli)
max_seq_length	int	No	Maximum input sequence length after tokenization (default: 128)
train_file	str	No	Path to custom CSV/JSON training data file
validation_file	str	No	Path to custom CSV/JSON validation data file
output_dir	str	Yes	Directory to save model checkpoints and predictions
do_train	bool	No	Whether to run training
do_eval	bool	No	Whether to run evaluation
do_predict	bool	No	Whether to run prediction on the test set

Outputs

Name	Type	Description
eval_results	dict	Evaluation metrics (accuracy, F1, correlation, etc.) for the task
predictions	numpy.ndarray	Model predictions for the test set
model_checkpoint	directory	Saved model weights, tokenizer, and training configuration
trainer_state	dict	Training state including loss history and learning rate schedule

Usage Examples

# Fine-tune BERT on SST-2 sentiment classification
# python run_glue.py \
#     --model_name_or_path bert-base-uncased \
#     --task_name sst2 \
#     --do_train \
#     --do_eval \
#     --max_seq_length 128 \
#     --per_device_train_batch_size 32 \
#     --learning_rate 2e-5 \
#     --num_train_epochs 3.0 \
#     --output_dir ./results/sst2

# Using a JSON config file
# python run_glue.py config.json

# Fine-tune on custom data
# python run_glue.py \
#     --model_name_or_path bert-base-uncased \
#     --train_file train.csv \
#     --validation_file validation.csv \
#     --do_train \
#     --do_eval \
#     --output_dir ./results/custom