Implementation:Hiyouga LLaMA Factory V1 Dtype Utils

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Machine Learning, Numerical Computing
Last Updated	2026-02-06 19:00 GMT

Overview

DtypeRegistry and DtypeInterface provide centralized precision dtype management, including alias resolution, hardware availability checking, and a context manager for temporarily changing the default PyTorch dtype.

Description

DtypeRegistry defines three class-level lists that map all recognized string aliases and torch.dtype values to their canonical precision groups: HALF_LIST (fp16/float16/half), FLOAT_LIST (fp32/float32/float), and BFLOAT_LIST (bf16/bfloat16). DtypeInterface uses HuggingFace's is_torch_fp16_available_on_device and is_torch_bf16_available_on_device utilities to detect hardware support at class initialization time. It provides static methods is_available, is_fp16, is_fp32, is_bf16, to_dtype (string-to-torch.dtype conversion), and to_str (torch.dtype-to-string conversion). The set_dtype context manager temporarily overrides the default PyTorch dtype and restores it upon exit. This module is inspired by Bytedance's verl library.

Usage

Use DtypeInterface to check whether a given precision is supported on the current hardware, convert between string and torch.dtype representations, or temporarily set the default dtype during model initialization. This is used throughout the v1 training and inference pipelines for mixed-precision configuration.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/v1/utils/dtype.py
• Lines: 1-91

Signature

class DtypeRegistry:
    HALF_LIST = ["fp16", "float16", "half", torch.float16]
    FLOAT_LIST = ["fp32", "float32", "float", torch.float32]
    BFLOAT_LIST = ["bf16", "bfloat16", torch.bfloat16]

class DtypeInterface:
    @staticmethod
    def is_available(precision: str | torch.dtype) -> bool: ...

    @staticmethod
    def is_fp16(precision: str | torch.dtype) -> bool: ...

    @staticmethod
    def is_fp32(precision: str | torch.dtype) -> bool: ...

    @staticmethod
    def is_bf16(precision: str | torch.dtype) -> bool: ...

    @staticmethod
    def to_dtype(precision: str | torch.dtype) -> torch.dtype: ...

    @staticmethod
    def to_str(precision: torch.dtype) -> str: ...

    @contextmanager
    def set_dtype(self, precision: str | torch.dtype): ...

Import

from llamafactory.v1.utils.dtype import DtypeRegistry, DtypeInterface

I/O Contract

Inputs

Name	Type	Required	Description
precision (is_available)	str or torch.dtype	Yes	Precision identifier string (e.g., "fp16", "bf16") or torch.dtype value
precision (to_dtype)	str or torch.dtype	Yes	Precision string or dtype to convert to canonical torch.dtype
precision (to_str)	torch.dtype	Yes	torch.dtype to convert to canonical string representation
precision (set_dtype)	str or torch.dtype	Yes	Precision to set as the temporary default torch dtype

Outputs

Name	Type	Description
is_available	bool	True if the given precision is supported on the current hardware device
is_fp16 / is_fp32 / is_bf16	bool	True if the precision matches the respective group
to_dtype	torch.dtype	Canonical torch.dtype (torch.float16, torch.float32, or torch.bfloat16)
to_str	str	Canonical string ("float16", "float32", or "bfloat16")
set_dtype	context manager	Yields with the default dtype temporarily changed, restores original on exit

Usage Examples

from llamafactory.v1.utils.dtype import DtypeInterface

# Check hardware availability
if DtypeInterface.is_available("bf16"):
    dtype = DtypeInterface.to_dtype("bf16")  # torch.bfloat16
else:
    dtype = DtypeInterface.to_dtype("fp32")  # torch.float32

# Convert dtype to string for logging
print(DtypeInterface.to_str(dtype))  # "bfloat16" or "float32"

# Temporarily set default dtype for model initialization
dtype_iface = DtypeInterface()
with dtype_iface.set_dtype("bf16"):
    model = MyModel()  # initialized with bfloat16 as default
# Original default dtype restored here

Related Pages

• Hiyouga_LLaMA_Factory_V1_Types - Type definitions used across the v1 module
• Hiyouga_LLaMA_Factory_V1_Logging - Logging utilities used alongside dtype configuration