Implementation:Axolotl ai cloud Axolotl RunPod Config Template
| Knowledge Sources | |
|---|---|
| Domains | Configuration, Deployment |
| Last Updated | 2026-02-07 00:00 GMT |
Overview
Comprehensive YAML configuration template that exposes all Axolotl training parameters as environment variable placeholders for RunPod serverless deployment.
Description
The config.yaml file in the RunPod integration directory serves as the master configuration template for deploying Axolotl training jobs on RunPod serverless infrastructure. It contains two parts: (1) an extensive commented reference section (lines 1-351) documenting every configuration option with descriptions and valid values, and (2) a parameterized template section (lines 354-567) where each field is set to an environment variable placeholder (e.g., ${BASE_MODEL}, ${LEARNING_RATE}). This allows RunPod request handlers to inject runtime parameters into a valid Axolotl configuration. The template covers model selection, quantization, dataset configuration, LoRA parameters, training hyperparameters, optimization, attention mechanisms, distributed training (FSDP/DeepSpeed), experiment tracking (W&B, MLflow, Comet, SwanLab), and special tokens.
Usage
Use this template when deploying Axolotl as a RunPod serverless endpoint. The RunPod handler reads incoming request parameters, substitutes the ${...} placeholders with actual values, and passes the resulting YAML to the Axolotl training pipeline. It also serves as a comprehensive configuration reference for all available Axolotl options.
Code Reference
Source Location
- Repository: Axolotl
- File: .runpod/src/config/config.yaml
- Lines: 1-567
Signature
# Reference section (lines 1-351): Commented documentation of all options
# Template section (lines 354-567): Parameterized config
base_model: ${BASE_MODEL}
load_in_4bit: ${LOAD_IN_4BIT}
datasets:
- path: ${DATASET_PATH}
type: ${DATASET_TYPE}
adapter: ${ADAPTER}
lora_r: ${LORA_R}
sequence_len: ${SEQUENCE_LEN}
learning_rate: ${LEARNING_RATE}
output_dir: ${OUTPUT_DIR}
# ... 100+ additional parameterized fields
Import
# Not imported directly. Used by RunPod handler:
# from axolotl.utils.config import prepare_config
# config = load_yaml_with_env_substitution(".runpod/src/config/config.yaml")
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| BASE_MODEL | str (env var) | Yes | HuggingFace model ID or local path |
| DATASET_PATH | str (env var) | Yes | HuggingFace dataset ID or local path |
| DATASET_TYPE | str (env var) | Yes | Prompt format type (alpaca, sharegpt, etc.) |
| OUTPUT_DIR | str (env var) | Yes | Directory to save trained model |
| All other ${} vars | various (env vars) | No | 100+ optional training parameters |
Outputs
| Name | Type | Description |
|---|---|---|
| Resolved YAML config | dict | Fully parameterized Axolotl configuration ready for training |
Usage Examples
Minimal RunPod Request
{
"input": {
"BASE_MODEL": "meta-llama/Meta-Llama-3-8B",
"DATASET_PATH": "tatsu-lab/alpaca",
"DATASET_TYPE": "alpaca",
"LOAD_IN_4BIT": "true",
"ADAPTER": "qlora",
"LORA_R": "16",
"SEQUENCE_LEN": "2048",
"LEARNING_RATE": "0.0002",
"NUM_EPOCHS": "3",
"OUTPUT_DIR": "/workspace/output"
}
}