Implementation:OpenGVLab InternVL LR Scheduler Builder

Knowledge Sources	OpenGVLab_InternVL
Domains	Learning Rate Scheduling, Training, Classification
Last Updated	2026-02-07 14:00 GMT

Overview

Constructs learning rate schedulers for the classification training pipeline, supporting cosine, linear, and step decay schedules with warmup periods.

Description

The build_scheduler factory function converts epoch-based configuration values (TRAIN.EPOCHS, TRAIN.WARMUP_EPOCHS, TRAIN.LR_SCHEDULER.DECAY_EPOCHS) into iteration-based step counts by multiplying with n_iter_per_epoch, then instantiates the appropriate scheduler:

• cosine -- Uses CosineLRScheduler from timm with a single cycle, warming up from WARMUP_LR to base LR, then decaying to MIN_LR.
• linear -- Uses a custom LinearLRScheduler class that linearly decays each parameter group's LR from its base value down to lr_min_rate * base_lr over the total training steps.
• step -- Uses StepLRScheduler from timm with a fixed decay rate applied at regular intervals.

The LinearLRScheduler class extends timm.scheduler.Scheduler and implements separate linear warmup and linear decay phases. It computes warmup step sizes per parameter group and supports both epoch-based and update-based stepping via get_epoch_values and get_update_values.

All schedulers operate in step mode (t_in_epochs=False) for per-iteration LR updates.

Usage

Use this module when setting up the training loop for InternViT classification. Call build_scheduler(config, optimizer, n_iter_per_epoch) after building the optimizer to obtain a scheduler that steps every iteration.

Code Reference

Source Location

• Repository: OpenGVLab_InternVL
• File: classification/lr_scheduler.py
• Lines: 1-111

Signature

def build_scheduler(config, optimizer, n_iter_per_epoch) -> Scheduler: ...

class LinearLRScheduler(Scheduler):
    def __init__(self, optimizer, t_initial, lr_min_rate, warmup_t=0,
                 warmup_lr_init=0., t_in_epochs=True, ...): ...
    def _get_lr(self, t) -> list: ...
    def get_epoch_values(self, epoch: int): ...
    def get_update_values(self, num_updates: int): ...

Import

from classification.lr_scheduler import build_scheduler

I/O Contract

Inputs

Name	Type	Required	Description
config	CfgNode	Yes	Configuration object with TRAIN.EPOCHS, TRAIN.WARMUP_EPOCHS, TRAIN.MIN_LR, TRAIN.WARMUP_LR, and TRAIN.LR_SCHEDULER settings
optimizer	torch.optim.Optimizer	Yes	The optimizer whose learning rate will be scheduled
n_iter_per_epoch	int	Yes	Number of training iterations per epoch

Outputs

Name	Type	Description
lr_scheduler	Scheduler	A timm-compatible learning rate scheduler instance

Usage Examples

Basic Usage

from classification.lr_scheduler import build_scheduler

lr_scheduler = build_scheduler(config, optimizer, n_iter_per_epoch=5000)

# Step the scheduler each iteration
for step in range(total_steps):
    lr_scheduler.step_update(step)

Related Pages

• Principle:OpenGVLab_InternVL_Learning_Rate_Scheduling