Implementation:Speechbrain Speechbrain Train RescueSpeech

Knowledge Sources	SpeechBrain
Domains	Speech_Enhancement, ASR
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for noise-robust speech recognition training on the RescueSpeech dataset provided by the SpeechBrain library.

Description

This recipe defines the ASR class (subclass of sb.core.Brain) for joint speech enhancement and recognition on the noisy RescueSpeech dataset. The pipeline combines an unfrozen SepFormer speech enhancement model (pre-fine-tuned on noisy RescueSpeech) with a Whisper encoder-decoder for ASR. Training is performed jointly, allowing both enhancement and ASR models to update their weights. The enhancement front-end produces cleaned speech that is fed into Whisper for recognition. Evaluation includes both ASR metrics (WER via greedy/beam search) and enhancement metrics (PESQ, STOI).

Usage

Use this recipe to train a noise-robust ASR system on the RescueSpeech dataset that jointly optimizes speech enhancement and recognition. Requires the RescueSpeech dataset with noisy and clean speech pairs and a pre-trained SepFormer checkpoint. Configure with hparams/robust_asr_16k.yaml.

Code Reference

Source Location

• Repository: SpeechBrain
• File: recipes/RescueSpeech/ASR/noise-robust/train.py

Signature

class ASR(sb.core.Brain):
    def compute_forward(self, batch, stage):
        ...
    def compute_forward_enhance(self, batch, stage):
        ...
    def compute_objectives(self, predictions, batch, stage):
        ...

Import

python recipes/RescueSpeech/ASR/noise-robust/train.py hparams/robust_asr_16k.yaml --data_folder /path/to/RescueSpeech

I/O Contract

Inputs

Name	Type	Required	Description
batch	PaddedBatch	Yes	Batch containing clean_sig, noisy_sig (waveforms), and tokens_bos (ASR tokens)
stage	sb.Stage	Yes	TRAIN, VALID, or TEST

Outputs

Name	Type	Description
predictions	tuple	Enhancement predictions, clean references, and ASR outputs (log_probs, hyps, wav_lens)
loss	torch.Tensor	Combined enhancement (SI-SNR) and ASR (NLL) loss

Usage Examples

python train.py hparams/robust_asr_16k.yaml --data_folder /path/to/RescueSpeech

Related Pages

• Principle:Speechbrain_Speechbrain_Conventional_Enhancement_Training