Implementation:OpenBMB UltraFeedback Conversation Get Prompt

Knowledge Sources	UltraFeedback FastChat
Domains	NLP, Prompt_Engineering
Last Updated	2023-10-02 00:00 GMT

Overview

Concrete tool for formatting prompts using the Conversation dataclass and model-specific templates adapted from FastChat.

Description

The fastchat.py module defines two core components:

SeparatorStyle (IntEnum): An enumeration of 14 separator styles controlling how roles, messages, and turns are delimited in the prompt string.

Conversation (dataclass): A prompt management class with fields for name, system prompt, roles, messages, separator style, separators, and stop criteria. The key method get_prompt() renders the accumulated messages into a formatted string using the configured separator style.

Six pre-defined template instances are provided and registered in the conv_template dictionary:

• llama → conv_llama2 (LLAMA2 style)
• alpaca → conv_alpaca (ADD_COLON_TWO style)
• vicuna → conv_vicuna_v1_1 (ADD_COLON_TWO style)
• wizardlm → conv_vicuna_v1_1 (same as vicuna)
• mpt → conv_mpt_chat (CHATML style)
• falcon → conv_falcon (RWKV style)

Usage

Import conv_template from fastchat and use conv_template[model_family].copy() to get a fresh template for each generation call. The template is mutable, so always copy before modifying.

Code Reference

Source Location

• Repository: UltraFeedback
• File: src/comparison_data_generation/fastchat.py (Lines 6-345)

Signature

class SeparatorStyle(IntEnum):
    ADD_COLON_SINGLE = auto()
    ADD_COLON_TWO = auto()
    ADD_COLON_SPACE_SINGLE = auto()
    NO_COLON_SINGLE = auto()
    NO_COLON_TWO = auto()
    ADD_NEW_LINE_SINGLE = auto()
    LLAMA2 = auto()
    CHATGLM = auto()
    CHATML = auto()
    CHATINTERN = auto()
    DOLLY = auto()
    RWKV = auto()
    PHOENIX = auto()
    ROBIN = auto()

@dataclasses.dataclass
class Conversation:
    name: str
    system: str
    roles: List[str]
    messages: List[List[str]]
    offset: int
    sep_style: SeparatorStyle
    sep: str
    sep2: str = None
    stop_str: str = None
    stop_token_ids: List[int] = None

    def get_prompt(self) -> str:
        """Renders all messages into a formatted prompt string
        based on the configured sep_style."""
        ...

    def append_message(self, role: str, message: str):
        """Appends a (role, message) pair to the conversation."""
        ...

    def copy(self) -> 'Conversation':
        """Returns a deep copy of this conversation template."""
        ...

conv_template: Dict[str, Conversation] = {
    "llama": conv_llama2,
    "alpaca": conv_alpaca,
    "vicuna": conv_vicuna_v1_1,
    "wizardlm": conv_vicuna_v1_1,
    "mpt": conv_mpt_chat,
    "falcon": conv_falcon,
}

Import

from fastchat import conv_template
# or: from fastchat import Conversation, SeparatorStyle, conv_template

I/O Contract

Inputs

Name	Type	Required	Description
model_family	str	Yes	Key into conv_template dict (llama, alpaca, vicuna, wizardlm, mpt, falcon)
system_prompt_suffix	str	No	Principle prompt appended to template's system field
instruction	str	Yes	User instruction text

Outputs

Name	Type	Description
prompt	str	Fully formatted prompt string ready for model inference
stop_str	Optional[str]	Stop string from the template (e.g., "\nUser" for Falcon)
stop_token_ids	Optional[List[int]]	Stop token IDs from the template

Usage Examples

Standard Usage with LLaMA-2

from fastchat import conv_template

# Get a fresh copy of the LLaMA-2 template
conv = conv_template["llama"].copy()

# Append principle prompt to system
principle_prompt = "The assistant should provide accurate and helpful information."
conv.system += " " + principle_prompt

# Add user instruction and empty assistant turn
conv.append_message(conv.roles[0], "Explain the theory of relativity.")
conv.append_message(conv.roles[1], None)

# Get formatted prompt
prompt = conv.get_prompt()
# Result: "<s>[INST] <<SYS>>\n...principle...\n<</SYS>>\n\nExplain the theory...[/INST]"

Special-Case Formatting (UltraLM)

# UltraLM uses custom formatting, not conv_template
if "ultralm" in model_type:
    system_prompt = ("User: A one-turn chat between a curious user and an "
                     "artificial intelligence assistant...</s>")
    system_prompt += "User: " + principle_prompt + "</s>"
    conv = [system_prompt]
    conv.append("User: " + instruction + "</s>")
    conv.append("Assistant: ")
    prompt = "\n".join(conv)

Related Pages

Implements Principle

• Principle:OpenBMB_UltraFeedback_Prompt_Formatting