Implementation:Facebookresearch Audiocraft DAdaptAdam

Knowledge Sources	Facebookresearch_Audiocraft D-Adaptation
Domains	Optimization, Training
Last Updated	2026-02-14 01:00 GMT

Overview

Concrete tool for Adam optimization with automatic D-Adaptation step-size selection that eliminates manual learning rate tuning.

Description

DAdaptAdam implements an Adam variant that automatically adapts the learning rate during training using the D-Adaptation method. The key insight is estimating D (the distance to the solution) online and using it to set the step size, removing the need for learning rate hyperparameter search.

Usage

Import this optimizer when you want to train AudioCraft models without manual learning rate tuning.

Code Reference

Source Location

• Repository: Facebookresearch_Audiocraft
• File: audiocraft/optim/dadam.py
• Lines: 1-248

Signature

class DAdaptAdam(torch.optim.Optimizer):
    def __init__(self, params, lr=1.0, betas=(0.9, 0.999), eps=1e-8,
                 weight_decay=0, log_every=0, decouple=True, d0=1e-6, growth_rate=float('inf')): ...
    def step(self, closure=None): ...

Import

from audiocraft.optim.dadam import DAdaptAdam

I/O Contract

Inputs

Name	Type	Required	Description
params	iterable	Yes	Model parameters to optimize
lr	float	No	Learning rate multiplier (default 1.0)
betas	tuple	No	Adam beta parameters (default (0.9, 0.999))

Outputs

Name	Type	Description
step result	None	Updates parameters in-place

Related Pages

• Principle:Facebookresearch_Audiocraft_D_Adaptation_Optimization