Implementation:Hiyouga LLaMA Factory WebUI Infer Component

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	WebUI, Inference, Gradio Components
Last Updated	2026-02-06 19:00 GMT

Overview

Gradio inference tab component that provides model loading/unloading controls and an embedded chat interface for interactive inference.

Description

The infer.py module defines create_infer_tab, which builds the inference tab with backend selection (huggingface, vllm, sglang), dtype dropdown (auto, float16, bfloat16, float32), and an extra arguments textbox. It includes load and unload buttons wired to the engine chatter's load_model and unload_model methods. An embedded chat box (from the chatbot component) is initially hidden and becomes visible once a model is successfully loaded. The multimodal input box visibility is dynamically toggled based on whether the selected model supports multimodal inputs, as determined by is_multimodal from the common module.

Usage

Use this component when assembling the main WebUI to provide real-time inference and chat capabilities. It is instantiated within create_ui in the interface module as the "Chat" tab.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/webui/components/infer.py
• Lines: 1-76

Signature

def create_infer_tab(engine: "Engine") -> dict[str, "Component"]: ...

Import

from llamafactory.webui.components.infer import create_infer_tab

I/O Contract

Inputs

Name	Type	Required	Description
engine	Engine	Yes	The WebUI engine instance providing access to Manager and WebChatModel (chatter)

Outputs

Name	Type	Description
elem_dict	dict[str, Component]	Dictionary of all named Gradio components: infer_backend, infer_dtype, extra_args, load_btn, unload_btn, info_box, plus all chat box elements (chat_box, role, system, tools, mm_box, image, video, audio, query, submit_btn, max_new_tokens, top_p, temperature, skip_special_tokens, escape_html, enable_thinking, clear_btn)

Usage Examples

# Adding the inference tab to the main UI
from llamafactory.webui.components.infer import create_infer_tab

with gr.Tab("Chat"):
    engine.manager.add_elems("infer", create_infer_tab(engine))

Related Pages

• Hiyouga_LLaMA_Factory_WebUI_Interface - Main interface that assembles the inference tab
• Hiyouga_LLaMA_Factory_WebUI_Chatbot - Chat box component embedded within the inference tab
• Hiyouga_LLaMA_Factory_WebUI_Engine - Engine providing the chatter for model load/unload
• Hiyouga_LLaMA_Factory_WebUI_Common - Provides is_multimodal for toggling multimodal input visibility