Jump to content

Connect SuperML | Leeroopedia MCP: Equip your AI agents with best practices, code verification, and debugging knowledge. Powered by Leeroo — building Organizational Superintelligence. Contact us at founders@leeroo.com.

Implementation:Hiyouga LLaMA Factory WebUI Common

From Leeroopedia


Knowledge Sources
Domains WebUI, Configuration, Utilities
Last Updated 2026-02-06 19:00 GMT

Overview

Shared utility module providing configuration management, path resolution, CLI command generation, and helper functions for the LLaMA Factory WebUI.

Description

The common.py module serves as the central utility layer for all other WebUI modules. It provides functions for persisting user configuration in YAML format (language, model hub, last model, custom paths), resolving model paths across multiple hubs (HuggingFace, ModelScope, OpenMind), managing checkpoint directory paths, and generating CLI commands for training previews. Additionally, it handles DeepSpeed configuration file creation (ZeRO stages 2 and 3 with optional offloading), process abortion via recursive signal propagation, dataset info loading from JSON, and evaluation result formatting.

Usage

Use this module when you need to load or save user preferences, resolve model paths, generate training CLI commands, create DeepSpeed configs, or perform any common WebUI operation. It is imported by virtually every other module in the WebUI package.

Code Reference

Source Location

Signature

def abort_process(pid: int) -> None: ...

def get_save_dir(*paths: str) -> os.PathLike: ...

def load_config() -> dict[str, str | dict[str, Any]]: ...

def save_config(
    lang: str, hub_name: str | None = None, model_name: str | None = None, model_path: str | None = None
) -> None: ...

def get_model_path(model_name: str) -> str: ...

def get_template(model_name: str) -> str: ...

def get_time() -> str: ...

def is_multimodal(model_name: str) -> bool: ...

def load_dataset_info(dataset_dir: str) -> dict[str, dict[str, Any]]: ...

def load_args(config_path: str) -> dict[str, Any] | None: ...

def save_args(config_path: str, config_dict: dict[str, Any]) -> None: ...

def gen_cmd(args: dict[str, Any]) -> str: ...

def save_cmd(args: dict[str, Any]) -> str: ...

def load_eval_results(path: os.PathLike) -> str: ...

def calculate_pixels(pixels: str) -> int: ...

def create_ds_config() -> None: ...

Import

from llamafactory.webui.common import (
    abort_process, get_save_dir, load_config, save_config,
    get_model_path, get_template, get_time, is_multimodal,
    load_dataset_info, load_args, save_args, gen_cmd, save_cmd,
    load_eval_results, calculate_pixels, create_ds_config,
    DEFAULT_CACHE_DIR, DEFAULT_CONFIG_DIR, DEFAULT_DATA_DIR, DEFAULT_SAVE_DIR,
)

I/O Contract

Inputs

Name Type Required Description
pid int Yes (abort_process) Process ID to terminate recursively
paths str (varargs) Yes (get_save_dir) Path segments joined under DEFAULT_SAVE_DIR
lang str Yes (save_config) Language code to persist in user config
hub_name str No (save_config) Model hub name ("huggingface", "modelscope", "openmind")
model_name str No (save_config) Model name to remember as last used model
model_path str No (save_config) Custom model path to associate with the model name
dataset_dir str Yes (load_dataset_info) Directory containing dataset_info.json
config_path str Yes (load_args/save_args) Path to YAML training configuration file
args dict[str, Any] Yes (gen_cmd/save_cmd) Training argument dictionary for CLI generation
pixels str Yes (calculate_pixels) Pixel expression, e.g. "224*224" or "50176"

Outputs

Name Type Description
config (load_config) dict[str, str or dict] User configuration with keys: lang, hub_name, last_model, path_dict, cache_dir
model_path (get_model_path) str Resolved filesystem or hub path for the given model name
template (get_template) str Template name for the model ("default" if not found)
time (get_time) str Current timestamp formatted as "YYYY-MM-DD-HH-MM-SS"
is_mm (is_multimodal) bool Whether the model supports multimodal inputs
dataset_info (load_dataset_info) dict Dataset metadata loaded from dataset_info.json
cmd_text (gen_cmd) str Formatted CLI command string wrapped in markdown code block
config_path (save_cmd) str Path to the saved training_args.yaml file

Usage Examples

# Load and save user configuration
from llamafactory.webui.common import load_config, save_config

config = load_config()
print(config["lang"])  # "en"
save_config(lang="zh", hub_name="modelscope", model_name="Qwen2-7B")
# Generate a CLI command preview for training arguments
from llamafactory.webui.common import gen_cmd

args = {"model_name_or_path": "meta-llama/Llama-3-8B", "output_dir": "saves/llama3"}
cmd = gen_cmd(args)
# Returns a markdown-formatted bash command string
# Create DeepSpeed config files
from llamafactory.webui.common import create_ds_config

create_ds_config()
# Creates ds_z2_config.json, ds_z2_offload_config.json,
# ds_z3_config.json, ds_z3_offload_config.json in llamaboard_cache/

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment