Implementation:Facebookresearch Audiocraft Export Directory Convention

Overview

Export_Directory_Convention is a pattern-level implementation documenting the expected file structure for AudioCraft model distribution. There is no single API that enforces this convention; rather, it emerges from the interaction between the export utilities and the loading infrastructure. This page documents the convention and the code that relies on it.

Implements

Principle:Facebookresearch_Audiocraft_Export_Directory_Organization

Pattern Documentation

This is a pattern doc -- it describes a file structure convention rather than a single function. The convention is enforced implicitly by the loader code in audiocraft/models/loaders.py, lines 40-71.

Expected File Structure

A complete model export directory should contain:

my_exported_model/
  state_dict.bin                  # Language model export (from export_lm)
  compression_state_dict.bin      # Compression model export (from export_pretrained_compression_model)

Loader Resolution Code

The _get_state_dict() function implements the resolution logic:

def _get_state_dict(
    file_or_url_or_id: Union[Path, str],
    filename: Optional[str] = None,
    device='cpu',
    cache_dir: Optional[str] = None,
):
    file_or_url_or_id = str(file_or_url_or_id)

    if os.path.isfile(file_or_url_or_id):
        return torch.load(file_or_url_or_id, map_location=device)

    if os.path.isdir(file_or_url_or_id):
        file = f"{file_or_url_or_id}/{filename}"
        return torch.load(file, map_location=device)

    elif file_or_url_or_id.startswith('https://'):
        return torch.hub.load_state_dict_from_url(
            file_or_url_or_id, map_location=device, check_hash=True)

    else:
        assert filename is not None
        file = hf_hub_download(
            repo_id=file_or_url_or_id,
            filename=filename,
            cache_dir=cache_dir,
            library_name="audiocraft",
            library_version=audiocraft.__version__,
        )
        return torch.load(file, map_location=device)

Source: audiocraft/models/loaders.py, lines 40-71

Filename Constants

The expected filenames are hardcoded in the specialized loader functions:

Loader Function	Expected Filename	Source Line
`load_lm_model_ckpt()`	`state_dict.bin`	loaders.py:L95
`load_compression_model_ckpt()`	`compression_state_dict.bin`	loaders.py:L75

Creating a Complete Export

The typical workflow to create a properly structured export directory:

from audiocraft.utils.export import export_lm, export_pretrained_compression_model
from pathlib import Path

export_dir = Path('/path/to/my_exported_model')

# Step 1: Export the language model
export_lm(
    checkpoint_path='/path/to/training/checkpoint_best.th',
    out_file=export_dir / 'state_dict.bin'
)

# Step 2: Export the compression model reference
export_pretrained_compression_model(
    pretrained_encodec='facebook/encodec_32khz',
    out_file=export_dir / 'compression_state_dict.bin'
)

HuggingFace Hub Upload

Once the export directory is created, it can be uploaded to HuggingFace Hub:

from huggingface_hub import HfApi

api = HfApi()
api.upload_folder(
    folder_path='/path/to/my_exported_model',
    repo_id='my-org/my-musicgen-model',
    repo_type='model',
)

After upload, the model can be loaded by anyone using:

from audiocraft.models import MusicGen
model = MusicGen.get_pretrained('my-org/my-musicgen-model')

Cache Directory

AudioCraft supports a custom cache directory for downloaded HuggingFace models via the AUDIOCRAFT_CACHE_DIR environment variable, resolved in get_audiocraft_cache_dir() at loaders.py:L36-37.

Related Implementations

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment