Implementation:Microsoft LoRA Legacy Utils Multiple Choice
Template:Implementation metadata
Overview
Utility module providing data structures and dataset processors for multiple choice reading comprehension tasks including RACE, SWAG, ARC, and Synonym.
Description
utils_multiple_choice.py is a utility library bundled within the HuggingFace Transformers legacy examples in the Microsoft LoRA repository. It defines frozen dataclasses for representing multiple choice examples and features, a base DataProcessor class with abstract methods for loading train/dev/test splits, and concrete processor subclasses for four benchmark datasets. It also provides a convert_examples_to_features function that tokenizes and encodes examples into model-ready tensors, plus MultipleChoiceDataset (PyTorch) and TFMultipleChoiceDataset (TensorFlow) dataset wrappers with distributed-safe caching via file locks.
The module belongs to the HuggingFace Transformers library (legacy examples), part of the modified fork shipped with Microsoft LoRA for NLU experiments.
⚠️ DEPRECATED: This file resides in the legacy/ directory and is not actively maintained. Prefer modern equivalents where available.
Usage
Use this module when you need to load and preprocess data for multiple choice reading comprehension tasks (RACE, SWAG, ARC, or Synonym) to fine-tune transformer models. Import the dataset classes directly if you are building a custom training loop, or use the processor classes to extract InputExample objects from raw data files.
Code Reference
Source Location
| Property | Value |
|---|---|
| File path | examples/NLU/examples/legacy/multiple_choice/utils_multiple_choice.py
|
| Lines | 579 |
| Module | utils_multiple_choice
|
Key Classes and Functions
| Name | Type | Signature / Description |
|---|---|---|
InputExample |
dataclass (frozen) | InputExample(example_id: str, question: str, contexts: List[str], endings: List[str], label: Optional[str])
|
InputFeatures |
dataclass (frozen) | InputFeatures(example_id: str, input_ids: List[List[int]], attention_mask: Optional[List[List[int]]], token_type_ids: Optional[List[List[int]]], label: Optional[int])
|
Split |
Enum | Values: train, dev, test
|
DataProcessor |
class | Abstract base class with get_train_examples(), get_dev_examples(), get_test_examples(), get_labels()
|
RaceProcessor |
class | Processor for RACE dataset (4 labels: 0-3), reads JSON from train/high and train/middle directories
|
SwagProcessor |
class | Processor for SWAG dataset (4 labels: 0-3), reads CSV files |
ArcProcessor |
class | Processor for ARC dataset (4 labels: 0-3), reads JSONL files, filters to 4-choice questions |
SynonymProcessor |
class | Processor for Synonym dataset (5 labels: 0-4), reads CSV files |
MultipleChoiceDataset |
class (PyTorch) | __init__(self, data_dir, tokenizer, task, max_seq_length, overwrite_cache, mode)
|
TFMultipleChoiceDataset |
class (TensorFlow) | TensorFlow equivalent with get_dataset() returning a tf.data.Dataset
|
convert_examples_to_features |
function | convert_examples_to_features(examples: List[InputExample], label_list: List[str], max_length: int, tokenizer: PreTrainedTokenizer) -> List[InputFeatures]
|
Import Usage
from utils_multiple_choice import (
MultipleChoiceDataset,
Split,
processors,
)
Processor Registry
processors = {
"race": RaceProcessor,
"swag": SwagProcessor,
"arc": ArcProcessor,
"syn": SynonymProcessor,
}
I/O Contract
Inputs
| Input | Type | Description |
|---|---|---|
data_dir |
str |
Path to directory containing dataset files (format varies by processor) |
tokenizer |
PreTrainedTokenizer |
HuggingFace tokenizer instance for encoding text |
task |
str |
Task name key into processors dict: "race", "swag", "arc", or "syn"
|
max_seq_length |
Optional[int] |
Maximum sequence length after tokenization (default None for PyTorch, 128 for TF) |
overwrite_cache |
bool |
Whether to overwrite cached features file (default False) |
mode |
Split |
Dataset split enum: Split.train, Split.dev, or Split.test
|
Outputs
| Output | Type | Description |
|---|---|---|
MultipleChoiceDataset[i] |
InputFeatures |
Single feature instance with input_ids, attention_mask, token_type_ids, label
|
convert_examples_to_features() return |
List[InputFeatures] |
List of tokenized, padded feature instances ready for model input |
| Cached features file | binary file | Saved at {data_dir}/cached_{mode}_{tokenizer_name}_{max_seq_length}_{task}
|
Usage Examples
Loading a RACE Dataset for Training
from transformers import AutoTokenizer
from utils_multiple_choice import MultipleChoiceDataset, Split
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
dataset = MultipleChoiceDataset(
data_dir="/path/to/RACE",
tokenizer=tokenizer,
task="race",
max_seq_length=512,
overwrite_cache=False,
mode=Split.train,
)
print(f"Number of examples: {len(dataset)}")
print(f"First feature input_ids shape: {len(dataset[0].input_ids)} choices x {len(dataset[0].input_ids[0])} tokens")
Converting Examples Directly
from utils_multiple_choice import RaceProcessor, convert_examples_to_features
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
processor = RaceProcessor()
examples = processor.get_train_examples("/path/to/RACE")
label_list = processor.get_labels() # ["0", "1", "2", "3"]
features = convert_examples_to_features(
examples=examples,
label_list=label_list,
max_length=512,
tokenizer=tokenizer,
)