Implementation:Speechbrain Speechbrain Train TimersAndSuch Wav2Vec

Knowledge Sources	SpeechBrain
Domains	SLU, Training
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for direct (speech to semantics) SLU training using wav2vec2 on the Timers-and-Such dataset provided by the SpeechBrain library.

Description

This recipe defines the SLU class (subclass of sb.Brain) for direct spoken language understanding on the Timers-and-Such dataset using wav2vec2 as the speech encoder. Input waveforms are encoded with a pre-trained wav2vec2 model (facebook/wav2vec2-base-960h from HuggingFace), then passed through an SLU encoder and seq2seq decoder to produce semantic token sequences. Supports waveform augmentation with label replication, beam search decoding, and semantic accuracy evaluation. The wav2vec2 encoder provides richer representations than the standard ASR transfer approach.

Usage

Use this recipe to train a direct SLU model on the Timers-and-Such dataset using wav2vec2 features. Requires the dataset and a pre-trained wav2vec2 model from HuggingFace. Configure with hparams/train_with_wav2vec2.yaml.

Code Reference

Source Location

• Repository: SpeechBrain
• File: recipes/timers-and-such/direct/train_with_wav2vec2.py

Signature

class SLU(sb.Brain):
    def compute_forward(self, batch, stage):
        ...
    def compute_objectives(self, predictions, batch, stage):
        ...

Import

python recipes/timers-and-such/direct/train_with_wav2vec2.py hparams/train_with_wav2vec2.yaml --data_folder /path/to/timers-and-such

I/O Contract

Inputs

Name	Type	Required	Description
batch	PaddedBatch	Yes	Batch containing sig (waveforms), tokens_bos, and tokens_eos (semantic tokens)
stage	sb.Stage	Yes	TRAIN, VALID, or TEST

Outputs

Name	Type	Description
predictions	tuple	Log-softmax sequence probabilities, wav_lens, and decoded semantic tokens
loss	torch.Tensor	Sequence-level NLL loss on semantic tokens

Usage Examples

python train_with_wav2vec2.py hparams/train_with_wav2vec2.yaml --data_folder /path/to/timers-and-such

Related Pages

• Principle:Speechbrain_Speechbrain_Spoken_Language_Understanding