Principle:Snorkel team Snorkel Multitask Training Execution

Knowledge Sources	An Overview of Multi-Task Learning in Deep Neural Networks
Domains	Training, Multi_Task_Learning, Optimization
Last Updated	2026-02-14 20:00 GMT

Overview

A training orchestration procedure that manages the multi-task optimization loop with configurable scheduling, logging, and checkpointing.

Description

Multitask Training Execution handles the complexity of training models with multiple tasks and dataloaders. Key considerations include:

• Batch scheduling: How to interleave batches from different task dataloaders (shuffled across tasks vs sequential per task)
• Loss aggregation: Computing and combining per-task losses
• Learning rate scheduling: Adjusting learning rates over training
• Gradient management: Gradient clipping to prevent exploding gradients
• Monitoring: Logging metrics and saving checkpoints

This is the same Trainer used for both general multi-task classification and slice-aware training.

Usage

Use this principle when training MultitaskClassifier models. The Trainer handles all training logistics; configure it through TrainerConfig parameters.

Theoretical Basis

The training loop follows:

For each epoch:

1. Draw batches according to batch_scheduler
2. For each batch from task $t$:
   1. Forward pass: ${\hat{y}}_t=\text{model}(x,t)$
   2. Compute loss: ${{ℒ}}_t={\text{loss}}_t({\hat{y}}_t,y_t)$
   3. Backward pass: $\mathrm{\nabla }\leftarrow \mathrm{\nabla }{{ℒ}}_t$
   4. Gradient clipping: $\Vert \mathrm{\nabla }\Vert \le \text{clip}$
   5. Optimizer step

Related Pages

Implemented By

• Implementation:Snorkel_team_Snorkel_Trainer_Fit_Multitask