Implementation:Hiyouga LLaMA Factory WebUI Chatter

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Web UI, Machine Learning
Last Updated	2026-02-06 19:00 GMT

Overview

WebChatModel extends ChatModel to provide a Gradio-integrated chat interface with model loading/unloading, streaming responses, thinking-block formatting, and multimodal input support.

Description

The WebChatModel class bridges the LLaMA-Factory chat inference engine with the Gradio WebUI. It supports lazy initialization (loading models on demand from UI controls) and demo mode (loading a preconfigured model from environment variables). The load_model method reads UI element values to construct model arguments, handles checkpoint paths for PEFT methods, configures quantization, and initializes the underlying ChatModel. The unload_model method clears the engine and triggers garbage collection. The stream method generates responses token-by-token, formatting output with HTML-escaped text, collapsible thinking blocks (via _format_response), and JSON-formatted tool calls. The append static method adds user messages to the chatbot history. Helper functions _escape_html and _format_response handle HTML safety and thinking-block display using <details> HTML elements. The update_attr context manager temporarily overrides object attributes (used to toggle enable_thinking on the template).

Usage

WebChatModel is instantiated by the WebUI Manager and wired to Gradio components. Use load_model to initialize inference from the UI, stream to generate streaming chat responses, and unload_model to release resources. This class should not typically be used outside the Gradio WebUI context.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/webui/chatter.py
• Lines: 1-246

Signature

def _escape_html(text: str) -> str: ...

def _format_response(text: str, lang: str, escape_html: bool, thought_words: tuple[str, str]) -> str: ...

@contextmanager
def update_attr(obj: Any, name: str, value: Any): ...

class WebChatModel(ChatModel):
    def __init__(self, manager: Manager, demo_mode: bool = False, lazy_init: bool = True) -> None: ...

    @property
    def loaded(self) -> bool: ...

    def load_model(self, data) -> Generator[str, None, None]: ...

    def unload_model(self, data) -> Generator[str, None, None]: ...

    @staticmethod
    def append(
        chatbot: list[dict[str, str]],
        messages: list[dict[str, str]],
        role: str,
        query: str,
        escape_html: bool,
    ) -> tuple[list[dict[str, str]], list[dict[str, str]], str]: ...

    def stream(
        self,
        chatbot: list[dict[str, str]],
        messages: list[dict[str, str]],
        lang: str,
        system: str,
        tools: str,
        image: Any | None,
        video: Any | None,
        audio: Any | None,
        max_new_tokens: int,
        top_p: float,
        temperature: float,
        skip_special_tokens: bool,
        escape_html: bool,
        enable_thinking: bool,
    ) -> Generator[tuple[list[dict[str, str]], list[dict[str, str]]], None, None]: ...

Import

from llamafactory.webui.chatter import WebChatModel

I/O Contract

Inputs

Name	Type	Required	Description
manager	Manager	Yes	WebUI Manager instance providing access to UI element mappings
demo_mode	bool	No	If True, loads model from DEMO_MODEL/DEMO_TEMPLATE environment variables
lazy_init	bool	No	If True (default), defers model loading until load_model is called
data (load_model)	dict	Yes	Gradio component data mapping UI element IDs to their current values
chatbot (stream)	list[dict[str, str]]	Yes	Current chatbot message history for the Gradio chatbot component
messages (stream)	list[dict[str, str]]	Yes	Internal message history in role/content format
lang	str	Yes	Language code for localized alert messages
system	str	Yes	System prompt text
tools	str	No	JSON string of available tool definitions
image / video / audio	Any or None	No	Multimodal input files from Gradio upload components
max_new_tokens	int	Yes	Maximum number of tokens to generate
top_p	float	Yes	Nucleus sampling probability threshold
temperature	float	Yes	Sampling temperature
skip_special_tokens	bool	Yes	Whether to skip special tokens in output
escape_html	bool	Yes	Whether to HTML-escape the output text
enable_thinking	bool	Yes	Whether to enable thinking/reasoning block display

Outputs

Name	Type	Description
loaded	bool	True if the engine is initialized and a model is loaded
load_model	Generator[str]	Yields status messages ("Loading...", "Loaded" or error messages)
unload_model	Generator[str]	Yields status messages ("Unloading...", "Unloaded" or error messages)
append	tuple[list, list, str]	Updated chatbot history, updated messages, and empty query string
stream	Generator[tuple[list, list]]	Yields (chatbot, messages) tuples as tokens are generated

Usage Examples

# Typical usage within the Gradio WebUI (not standalone)
from llamafactory.webui.chatter import WebChatModel

# Initialize with a manager instance
chat_model = WebChatModel(manager=manager, demo_mode=False, lazy_init=True)

# Load model from UI data (called by Gradio event handler)
for status in chat_model.load_model(data):
    print(status)  # "Loading...", "Model loaded successfully"

# Append a user message
chatbot, messages, _ = WebChatModel.append(
    chatbot=[], messages=[], role="user", query="Hello!", escape_html=True
)

# Stream a response (called by Gradio event handler)
for chatbot, messages in chat_model.stream(
    chatbot, messages, lang="en", system="You are helpful.",
    tools="", image=None, video=None, audio=None,
    max_new_tokens=512, top_p=0.9, temperature=0.7,
    skip_special_tokens=True, escape_html=True, enable_thinking=False,
):
    pass  # Gradio updates the UI with each yield

Related Pages

• Hiyouga_LLaMA_Factory_V1_Rendering_Plugin - Rendering plugin system for message template formatting
• Hiyouga_LLaMA_Factory_V1_Types - Type definitions for Message and content structures
• Hiyouga_LLaMA_Factory_V1_CLI_Sampler - CLI-based alternative for interactive model chat