Implementation:Mit han lab Llm awq Get prompter

Overview

Concrete tool for selecting the correct prompt template and stop tokens for LLM chat provided by the llm-awq library.

Source

tinychat/utils/prompt_templates.py, Lines 343-399

Signature

def get_prompter(model_type, model_path="", short_prompt=False, empty_prompt=False):

def get_stop_token_ids(model_type, model_path=""):

Import

from tinychat.utils.prompt_templates import get_prompter, get_stop_token_ids

I/O

get_prompter

Inputs:

• model_type (str) - one of "llama", "falcon", "mpt", "qwen", "nvila", "internvl3"
• model_path (str) - used to detect variants like vicuna, llama-3, llava, vila
• short_prompt (bool) - whether to use a shortened system prompt
• empty_prompt (bool) - whether to use an empty system prompt

Output:

• A BasePrompter subclass instance with insert_prompt() and update_template() methods

get_stop_token_ids

Inputs:

• model_type (str)
• model_path (str)

Output:

• List[int] of stop token IDs specific to the detected model variant

Supported Prompters

• Llama2Prompter - for LLaMA-2 chat models using [INST]/[/INST] format
• Llama3Prompter - for LLaMA-3 models using header_id tags
• VicunaPrompter - for Vicuna models using USER:/ASSISTANT: format
• LlavaLlamaPrompter - for LLaVA multimodal models
• FalconSimplePrompter - for Falcon instruction models
• MPTChatPrompter - for MPT chat models
• QwenPrompter - for Qwen models
• NVILAPrompter - for NVILA vision-language models
• InternVL3Prompter - for InternVL3 models

Related Pages

• Principle:Mit_han_lab_Llm_awq_Prompt_Template_Configuration

Knowledge Sources

• Repo|llm-awq|https://github.com/mit-han-lab/llm-awq

Domains

• NLP
• Deployment