Implementation:Speechbrain Speechbrain Normalize Util

Knowledge Sources	SpeechBrain
Domains	Text_Processing, ASR
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for text normalization utilities used during ASR inference on Switchboard data provided by the SpeechBrain library.

Description

This module provides utility functions for normalizing predicted and target word sequences during ASR model inference, specifically tailored for the Switchboard corpus conventions. The core function normalize_words performs several transformations: (1) removes incomplete words (those starting with "-"), (2) expands common English contractions (e.g., "won't" to "WILL NOT", "can't" to "CAN NOT") using extensive regex-based rules, (3) splits hyphen-linked words, (4) checks predictions against a GLM (Global Mapping) alternatives file that maps alternate spellings and contractions from the ARPA Hub4-E and Hub5-E standards, and (5) filters out filler words and hesitations (e.g., "UH", "UM", "MM", "HM", "AH") from an exclusion list. The read_glm_csv function loads the alternatives dictionary from a CSV file. All processing assumes UPPERCASE text throughout the recipe.

Usage

Import normalize_words after decoding and tokenization in ASR pipelines to clean up predictions and references before computing WER. This is adapted from Kaldi's local/score.sh normalization procedure.

Code Reference

Source Location

• Repository: SpeechBrain
• File: recipes/Switchboard/ASR/CTC/normalize_util.py

Signature

def read_glm_csv(save_folder):
    """Load the ARPA Hub4-E and Hub5-E alternate spellings and contractions map."""
    ...

def expand_contractions(text) -> list:
    """Expand common contractions and split linked words."""
    ...

def expand_contractions_batch(text_batch):
    """Wrapper that handles a batch of words for contraction expansion."""
    ...

def normalize_words(target_words_batch, predicted_words_batch, glm_alternatives=None):
    """Remove references and hypotheses we don't want to score.
    Expands contractions, applies GLM alternatives, and filters exclusions."""
    ...

Import

from normalize_util import normalize_words, read_glm_csv

I/O Contract

Inputs

Name	Type	Required	Description
target_words_batch	list[list[str]]	Yes	Batch of target word lists (one list per utterance)
predicted_words_batch	list[list[str]]	Yes	Batch of predicted word lists (one list per utterance)
glm_alternatives	dict	No	Dictionary of valid word alternatives from GLM file
save_folder	str	Yes (read_glm_csv)	Path to folder containing glm.csv
text	str	Yes (expand_contractions)	Single text string to expand

Outputs

Name	Type	Description
checked_predicted_words_batch	list[list[str]]	Normalized predicted words with contractions expanded and exclusions removed
target_words_batch	list[list[str]]	Normalized target words with contractions expanded
alternatives_dict	dict	Loaded GLM alternatives mapping (from read_glm_csv)

Usage Examples

from normalize_util import normalize_words, read_glm_csv

# Load GLM alternatives
glm_alternatives = read_glm_csv("/path/to/save_folder")

# Normalize predictions and targets
target_batch = [["I", "WON'T", "GO"]]
predicted_batch = [["I", "WONT", "GO", "UM"]]

norm_targets, norm_preds = normalize_words(
    target_batch, predicted_batch, glm_alternatives=glm_alternatives
)
# norm_targets: [["I", "WILL", "NOT", "GO"]]
# norm_preds: [["I", "WILL", "NOT", "GO"]]  (UM excluded)

Related Pages

• Principle:Speechbrain_Speechbrain_ASR_Evaluation_With_WER