Implementation:Hiyouga LLaMA Factory API Protocol

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	API, Data Models
Last Updated	2026-02-06 19:00 GMT

Overview

API Protocol defines the complete set of Pydantic data models for the OpenAI-compatible chat completions API protocol used by LLaMA Factory.

Description

The module provides type-safe enums for Role (user, assistant, system, function, tool) and Finish reasons (stop, length, tool_calls), along with Pydantic BaseModel classes covering the full OpenAI chat completion specification. This includes request schemas (ChatCompletionRequest, ScoreEvaluationRequest), response schemas (ChatCompletionResponse, ChatCompletionStreamResponse, ScoreEvaluationResponse), multimodal input items (text, image_url, video_url, audio_url via MultimodalInputItem), function calling models (Function, FunctionDefinition, FunctionAvailable, FunctionCall), and model listing (ModelCard, ModelList).

Usage

Use these models when building or consuming requests and responses for the LLaMA Factory API. They are imported by app.py for route type annotations and by chat.py for response construction.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/api/protocol.py
• Lines: 1-156

Signature

class Role(StrEnum):
    USER = "user"
    ASSISTANT = "assistant"
    SYSTEM = "system"
    FUNCTION = "function"
    TOOL = "tool"

class Finish(StrEnum):
    STOP = "stop"
    LENGTH = "length"
    TOOL = "tool_calls"

class ChatCompletionRequest(BaseModel):
    model: str
    messages: list[ChatMessage]
    tools: list[FunctionAvailable] | None = None
    do_sample: bool | None = None
    temperature: float | None = None
    top_p: float | None = None
    n: int = 1
    presence_penalty: float | None = None
    max_tokens: int | None = None
    stop: str | list[str] | None = None
    stream: bool = False

class ChatCompletionResponse(BaseModel):
    id: str
    object: Literal["chat.completion"] = "chat.completion"
    created: int
    model: str
    choices: list[ChatCompletionResponseChoice]
    usage: ChatCompletionResponseUsage

class ScoreEvaluationRequest(BaseModel):
    model: str
    messages: list[str]
    max_length: int | None = None

class ScoreEvaluationResponse(BaseModel):
    id: str
    object: Literal["score.evaluation"] = "score.evaluation"
    model: str
    scores: list[float]

Import

from llamafactory.api.protocol import (
    ChatCompletionRequest,
    ChatCompletionResponse,
    Role,
    Finish,
    ScoreEvaluationRequest,
    ScoreEvaluationResponse,
)

I/O Contract

Inputs

Name	Type	Required	Description
model	str	Yes	Model identifier string
messages	list[ChatMessage]	Yes	List of chat messages with role and content
tools	list[FunctionAvailable]	No	Available tool/function definitions
temperature	float	No	Sampling temperature
top_p	float	No	Nucleus sampling parameter
n	int	No	Number of completions to generate (default: 1)
max_tokens	int	No	Maximum number of tokens to generate
stream	bool	No	Whether to stream the response (default: False)

Outputs

Name	Type	Description
ChatCompletionResponse	BaseModel	Complete chat response with choices, usage, and metadata
ChatCompletionStreamResponse	BaseModel	Streaming chunk with delta content
ScoreEvaluationResponse	BaseModel	Reward model scores for input messages
ModelList	BaseModel	List of available model cards

Usage Examples

from llamafactory.api.protocol import ChatCompletionRequest, ChatMessage, Role

# Construct a chat completion request
request = ChatCompletionRequest(
    model="llama-2-7b",
    messages=[
        ChatMessage(role=Role.USER, content="Hello, how are you?")
    ],
    temperature=0.7,
    max_tokens=256,
)

Related Pages

• Hiyouga_LLaMA_Factory_API_App - FastAPI app that uses these protocol models for route definitions
• Hiyouga_LLaMA_Factory_API_Chat - Chat functions that construct responses using these models
• Hiyouga_LLaMA_Factory_Base_Engine - Engine interface whose Response dataclass maps to these protocol types