Principle:Deepset ai Haystack Chat Prompt Templating
Overview
Chat prompt templating constructs multi-turn chat messages (system, user, assistant) from templates, enabling structured conversation prompts for chat-based LLMs. It extends the concept of prompt templating to the chat completion paradigm, where prompts are sequences of role-tagged messages rather than a single flat string.
Domains
- NLP
- Prompt_Engineering
Theory
Chat prompt templating extends template-based prompt construction to the chat message format with role-tagged messages. It supports both list-of-ChatMessage and string template formats with Jinja2 variables.
Role-Based Message Structure
Modern chat LLMs expect input as a sequence of messages, each tagged with a role:
- System: Sets the overall behavior, persona, or constraints for the assistant. Typically the first message.
- User: Contains the human input, question, or request.
- Assistant: Represents prior model responses, used for few-shot examples or multi-turn context.
This role-based structure allows more nuanced control over LLM behavior than flat prompt strings. For example, system messages can enforce output format, language, or persona, while user and assistant messages establish conversational context.
Template Formats
Chat prompt templating supports two complementary template formats:
List-of-ChatMessage Format
Templates are expressed as Python lists of ChatMessage objects, each containing Jinja2 template syntax in their text content:
template = [
ChatMessage.from_system("You are a helpful assistant speaking {{ language }}."),
ChatMessage.from_user("Tell me about {{ topic }}.")
]
This format is explicit about message boundaries and roles, and preserves any non-text content or metadata attached to the messages.
String Template Format
Templates use a special Jinja2 extension with {% message role="..." %} block tags:
template = """
{% message role="system" %}
You are a helpful assistant.
{% endmessage %}
{% message role="user" %}
Tell me about {{ topic }}.
{% endmessage %}
"""
The string format supports the templatize_part filter for embedding non-text content (such as images) within messages, which is not possible with the list-of-ChatMessage format.
Variable Interpolation in Chat Context
Within each message, Jinja2 variables are interpolated the same way as in flat prompt templating. However, variable rendering only occurs in user and system messages. Assistant messages are passed through unmodified, preserving the integrity of prior model outputs or handcrafted few-shot examples.
Design Benefits
- Structured conversation control: Precise control over the role and ordering of messages.
- Dynamic multi-turn prompts: Templates can be swapped at runtime, enabling dynamic conversation flows.
- Multi-modal support: String template format supports embedding images and other non-text parts via the
templatize_partfilter. - Reusability: Chat prompt templates can be parameterized and reused across different conversational scenarios.