Implementation:Facebookresearch Audiocraft GLU Activations

Knowledge Sources	Facebookresearch_Audiocraft
Domains	Neural_Networks, Activations
Last Updated	2026-02-14 01:00 GMT

Overview

Concrete tool for custom Gated Linear Unit activation variants and a factory helper function for use in AudioCraft neural network modules.

Description

This module defines CustomGLU variants (SiGLU, ReGLU, GeGLU) that apply gating with different activation functions (Sigmoid, ReLU, GELU). The get_activation_fn factory function instantiates activations by name, supporting both standard PyTorch activations and the custom GLU variants.

Usage

Import get_activation_fn when configuring activation functions for transformer or codec modules by name string.

Code Reference

Source Location

• Repository: Facebookresearch_Audiocraft
• File: audiocraft/modules/activations.py
• Lines: 1-96

Signature

class CustomGLU(nn.Module):
    def __init__(self, activation: nn.Module, dim: int = -1): ...
    def forward(self, x: torch.Tensor) -> torch.Tensor: ...

def get_activation_fn(activation: str) -> tp.Union[str, tp.Callable]:
    """Return activation function by name: 'relu', 'gelu', 'siglu', 'reglu', 'geglu'."""

Import

from audiocraft.modules.activations import get_activation_fn

I/O Contract

Inputs

Name	Type	Required	Description
activation	str	Yes	Activation name ('relu', 'gelu', 'siglu', 'reglu', 'geglu')

Outputs

Name	Type	Description
activation_fn	Callable	Activation function or module

Related Pages

• Principle:Facebookresearch_Audiocraft_Autoregressive_Token_Generation