Implementation:Hiyouga LLaMA Factory Kernel Interface

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Machine Learning, Hardware Acceleration
Last Updated	2026-02-06 19:00 GMT

Overview

Kernel plugin discovery, registration, and application interface that orchestrates hardware-specific kernel optimizations for model operations in LLaMA-Factory's v1 plugin system.

Description

interface.py is the top-level orchestrator for the kernel optimization pipeline in LLaMA-Factory's v1 architecture. It provides:

• scan_all_kernels: Dynamically discovers and imports all Python modules under the ops/ directory structure. Importing these modules triggers the @register_kernel decorator, which automatically registers kernel implementations in the global Registry. This function runs at module load time, populating the default_kernels dictionary.
• get_default_kernels: Returns a list of all registered kernel IDs available for the current environment.
• apply_kernel: Applies a specific kernel by ID to a model by calling the kernel's apply() method.
• KernelPlugin: A plugin class (extending BasePlugin) that wraps the kernel interface for integration with the v1 plugin system.
• apply_default_kernels: The main plugin entry point (registered as KernelPlugin("auto")) that applies all or a subset of registered kernels to a model. Supports "auto" mode (apply all), explicit comma-separated kernel IDs, or disabled (None/False).

The architecture enables transparent, hardware-specific optimizations (NPU, GPU, etc.) without requiring changes to model code.

Usage

Use this module when loading models in the v1 pipeline to automatically apply hardware-optimized kernels. The plugin system calls apply_default_kernels during model preparation. Individual kernels can also be applied selectively by specifying kernel IDs.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/v1/plugins/model_plugins/kernels/interface.py
• Lines: 1-140

Signature

def scan_all_kernels() -> dict[str, type[BaseKernel]]

def get_default_kernels() -> list[str]

def apply_kernel(kernel_id: str, **kwargs) -> None

class KernelPlugin(BasePlugin):
    pass

@KernelPlugin("auto").register()
def apply_default_kernels(model: HFModel, include_kernels: str = None) -> HFModel

Import

from llamafactory.v1.plugins.model_plugins.kernels.interface import apply_default_kernels
from llamafactory.v1.plugins.model_plugins.kernels.interface import apply_kernel
from llamafactory.v1.plugins.model_plugins.kernels.interface import get_default_kernels
from llamafactory.v1.plugins.model_plugins.kernels.interface import KernelPlugin

I/O Contract

Inputs

apply_default_kernels

Name	Type	Required	Description
model	HFModel	Yes	The HuggingFace model instance to apply kernels to
include_kernels	str	No	Comma-separated kernel IDs to apply; "auto" or True applies all defaults; None or False disables kernel application

apply_kernel

Name	Type	Required	Description
kernel_id	str	Yes	The registered ID of the kernel to apply (e.g., "npu_fused_swiglu", "npu_fused_rope")
**kwargs	Any	No	Keyword arguments passed to the kernel's apply method; typically includes model=model

Outputs

apply_default_kernels

Name	Type	Description
model	HFModel	The model with applied kernel optimizations (monkey-patched forward methods)

scan_all_kernels

Name	Type	Description
kernels	dict[str, type[BaseKernel]]	Dictionary mapping kernel IDs to their registered kernel classes

Usage Examples

# Apply all available kernels automatically
from llamafactory.v1.plugins.model_plugins.kernels.interface import apply_default_kernels

model = apply_default_kernels(model, include_kernels="auto")

# Apply a specific kernel
from llamafactory.v1.plugins.model_plugins.kernels.interface import apply_kernel

apply_kernel("npu_fused_swiglu", model=model)

# List available kernels
from llamafactory.v1.plugins.model_plugins.kernels.interface import get_default_kernels

kernel_ids = get_default_kernels()
# e.g., ["npu_fused_swiglu", "npu_fused_rope", "npu_fused_moe"]

Related Pages

• Hiyouga_LLaMA_Factory_NPU_SwiGLU - NPU SwiGLU kernel registered through this interface
• Hiyouga_LLaMA_Factory_NPU_RoPE - NPU RoPE kernel registered through this interface
• Hiyouga_LLaMA_Factory_NPU_Fused_MoE - NPU Fused MoE kernel registered through this interface