Implementation:Neuml Txtai RAG Prompt Template

Knowledge Sources	txtai txtai Documentation
Domains	NLP, RAG
Last Updated	2026-02-10 00:00 GMT

Overview

Concrete tool for designing prompt templates that combine user questions with retrieved context for LLM generation provided by the txtai library.

Description

The RAG prompt template is a user-defined string pattern that the RAG pipeline uses to construct the final input sent to the generative model. The template must contain {question} and {context} placeholders, which are substituted at runtime with the user's query and the concatenated text of retrieved passages, respectively.

When no custom template is provided, the RAG pipeline defaults to "{question} {context}" -- a simple concatenation. For production use, more structured templates are strongly recommended (e.g., including explicit instructions for the model to answer based on the provided context).

An optional system prompt can be specified separately, also supporting {question} and {context} placeholders. When a system prompt is set, the pipeline constructs a message list with "system" and "user" roles rather than a single prompt string, which is the expected format for chat-oriented LLMs.

Usage

Use this pattern when configuring a RAG pipeline to control how the generative model receives its input. Templates are passed as the template parameter to RAG.__init__ and system prompts as the system parameter.

Interface Specification

Source Location

• Repository: txtai
• File: src/python/txtai/pipeline/llm/rag.py
• Lines: 84-85 (default template), 329-356 (prompts method)

Template Pattern

The template is a Python format string with two required placeholders:

# Required placeholders: {question} and {context}
template = "Answer the following question using the context below.\n\nContext: {context}\n\nQuestion: {question}\n\nAnswer:"

System Prompt Pattern

The optional system prompt follows the same placeholder convention:

# Optional system prompt with the same placeholders
system = "You are a helpful assistant. Answer questions using only the provided context. If the context does not contain the answer, say so."

Default Template

# Default when no template is specified (rag.py, line 85)
template = "{question} {context}"

Prompt Construction Logic

The prompts method in RAG (lines 329-356) constructs prompts as follows:

def prompts(self, questions, contexts):
    prompts = []
    for x, context in enumerate(contexts):
        # Substitute placeholders in user template
        prompt = self.template.format(question=questions[x], context=context)

        # If system prompt is defined, create message list
        if self.system:
            prompt = [
                {"role": "system", "content": self.system.format(question=questions[x], context=context)},
                {"role": "user", "content": prompt},
            ]

        prompts.append(prompt)

    return prompts

Context Separator

Retrieved passages are joined with a configurable separator before being inserted into the {context} placeholder. The separator defaults to a single space " " and is set via the separator parameter in RAG.__init__.

I/O Contract

Inputs

Name	Type	Required	Description
template	str	No	A Python format string containing {question} and {context} placeholders. Defaults to "{question} {context}".
system	str	No	An optional system prompt format string. May contain {question} and {context} placeholders. Defaults to None.
separator	str	No	String used to join context passages before template substitution. Defaults to " ".

Outputs

Name	Type	Description
prompt	str or list of dict	When no system prompt is set, a plain string with placeholders substituted. When a system prompt is set, a list of message dicts with "role" and "content" keys.

Usage Examples

Basic Example

from txtai import Embeddings
from txtai.pipeline import RAG

embeddings = Embeddings({"content": True})
embeddings.index([
    (0, "The speed of light is approximately 299,792 km/s.", None),
    (1, "Sound travels at about 343 m/s in air at room temperature.", None),
])

# Simple template
rag = RAG(
    embeddings,
    "google/flan-t5-base",
    template="Answer the question using the context.\n\nContext: {context}\n\nQuestion: {question}"
)
answer = rag("How fast is light?")

Template with System Prompt

from txtai import Embeddings
from txtai.pipeline import RAG

embeddings = Embeddings({"content": True})
embeddings.index([
    (0, "Python was created by Guido van Rossum and first released in 1991.", None),
    (1, "Java was developed by James Gosling at Sun Microsystems.", None),
])

# Structured template with system prompt for chat-oriented LLMs
rag = RAG(
    embeddings,
    "meta-llama/Meta-Llama-3-8B-Instruct",
    template="Based on the following context, answer the question.\n\nContext:\n{context}\n\nQuestion: {question}\n\nAnswer:",
    system="You are a precise assistant. Only use information from the provided context. If the context does not contain the answer, respond with 'I don't know.'",
    separator="\n\n"
)
answer = rag("Who created Python?")

Custom Separator for Numbered Context

from txtai import Embeddings
from txtai.pipeline import RAG

embeddings = Embeddings({"content": True})
embeddings.index([
    (0, "Fact A: The Earth is the third planet from the Sun.", None),
    (1, "Fact B: Mars is known as the Red Planet.", None),
    (2, "Fact C: Jupiter is the largest planet in the Solar System.", None),
])

# Use newline separator for clearer passage boundaries
rag = RAG(
    embeddings,
    "google/flan-t5-large",
    template="Use the following facts to answer the question.\n\nFacts:\n{context}\n\nQuestion: {question}\n\nAnswer:",
    separator="\n"
)
answer = rag("Which planet is the largest?")

Related Pages

Implements Principle

• Principle:Neuml_Txtai_Prompt_Engineering