Principle:Microsoft BIPIA Prompt Construction
| Field | Value |
|---|---|
| Sources | BIPIA paper |
| Domains | NLP, Prompt_Engineering, Security |
| Last Updated | 2026-02-14 |
Overview
A prompt formatting methodology that transforms poisoned dataset examples into model-specific input formats, handling both system/user prompt separation and chat/completion template variations.
Description
The prompt construction process operates in two distinct phases that decouple task-specific prompt generation from model-specific formatting.
Phase 1: Task-Specific Prompt Generation. The method BasePIABuilder.construct_prompt() creates task-specific prompts from dataset examples. Each task builder (for example, EmailIPIABuilder, QAIPIADataset, or SummIPIABuilder) implements this abstract method to produce prompts appropriate for its domain. When the target model supports system prompts (require_system_prompt=True), the method returns a tuple of (system_prompt, user_prompt), separating the instruction context from the user query. When the model does not support system prompts, it returns a single combined string.
Phase 2: Model-Specific Formatting. The method BaseModel.process_fn() takes the output of phase 1 and formats it into the structure expected by the target model's API. For OpenAI chat models, this means constructing a list of message dictionaries with system and user roles. For open-source models served through FastChat, it means populating a conversation template with the appropriate special tokens and role prefixes. For completion-style models, it means assembling raw text with the correct delimiters.
This two-layer design ensures that task logic and model logic remain independent. Adding a new task requires only implementing construct_prompt(); adding a new model requires only implementing process_fn().
Usage
Use prompt construction when converting raw dataset examples into formatted inputs suitable for different LLM APIs. The two-layer design ensures the same dataset can be evaluated across models with different prompt formats. A single benchmark dataset can be run against OpenAI chat models, HuggingFace models with FastChat templates, and vLLM-served models without changing any task-level code.
Theoretical Basis
The design follows the principle of separation of concerns: task builders handle content (what to ask the model), while model wrappers handle format (how to present that content to the model).
This separation mirrors the distinction between an abstract interface and its concrete serialization. The task builder defines the semantic payload; the model wrapper defines the wire format.
Pseudocode of the flow:
dataset_example
-> construct_prompt(example)
-> (system_prompt, user_prompt) # or a single combined string
-> model.process_fn(example, construct_prompt)
-> formatted message # e.g. list[dict] for chat API, str for completion API
In this flow, construct_prompt is passed as a callable into process_fn, allowing the model wrapper to invoke it with knowledge of whether a system prompt is needed. This inversion of control keeps the two layers loosely coupled: the model wrapper decides when to call the prompt constructor and how to use its output, but never needs to know what the prompt contains.