Implementation:OpenBMB UltraFeedback Principle Selection
| Knowledge Sources | |
|---|---|
| Domains | NLP, Alignment |
| Last Updated | 2023-10-02 00:00 GMT |
Overview
Concrete tool for selecting principle categories and system prompts based on dataset subset, as implemented in the UltraFeedback generation pipeline.
Description
The principle selection logic is implemented in two locations: inline within instruction_completion in main.py (HuggingFace backend) and as a standalone sample_principle function in main_vllm.py (vLLM backend). Both implementations share the same principles dictionary containing ~55 prompt variants across 5 categories.
The vLLM version (sample_principle) also performs prompt formatting in the same function, combining principle selection with the Conversation template system to produce a ready-to-use prompt string stored in example["prompt"].
Usage
This logic is called once per instruction-model pair during the generation phase. It is invoked via dataset.map(instruction_completion) in the HF backend or dataset.map(sample_principle) in the vLLM backend.
Code Reference
Source Location
- Repository: UltraFeedback
- File: src/comparison_data_generation/main.py (Lines 34-88 for principles dict, Lines 162-180 for selection logic)
- File: src/comparison_data_generation/main_vllm.py (Lines 29-84 for principles dict, Lines 105-128 for sample_principle function)
Signature
# Principles dictionary (shared between both backends)
principles = {
"helpfulness": [
"The assistant should provide users with accurate, relevant, and up-to-date information...",
# ... 11 variants total
],
"harmlessness": [# 11 variants],
"honesty": [# 11 variants],
"verbalized_calibration": [# 1 variant],
"truthfulness": [# 11 variants],
}
# vLLM backend: standalone function
def sample_principle(example: Dict) -> Dict:
"""
Selects a principle category and prompt based on the global subset variable.
Also constructs the formatted prompt and appends a partial completion dict.
Args:
example: Dict with 'instruction', 'models', 'completions' fields
Returns:
example: Dict with added 'prompt' field and appended completion stub
"""
...
# HF backend: inline within instruction_completion (main.py:L162-180)
# Same selection logic, but does not create example["prompt"]
Import
import random
import numpy as np
# principles dict is defined at module level in main.py / main_vllm.py
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| example | Dict | Yes | Dataset example with 'instruction', 'models', 'completions' fields |
| subset | str | Yes | Global variable: dataset subset name determining principle distribution |
| model_type | str | Yes | Global variable: current model being processed (for filtering) |
Outputs
| Name | Type | Description |
|---|---|---|
| principle | str | Selected category name (helpfulness, harmlessness, honesty, truthfulness, verbalized_calibration) |
| principle_prompt | str | Selected system prompt text from the category |
| example["prompt"] | str | (vLLM only) Formatted prompt string ready for inference |
| example["completions"] | List[Dict] | Appended with partial completion dict containing model, principle, custom_system_prompt |
Usage Examples
Principle Selection Logic (Extracted)
import random
import numpy as np
# Subset-conditional principle selection
subset = "sharegpt"
if subset in ["sharegpt", "ultrachat"]:
principle = random.choice(
["helpfulness", "helpfulness", "helpfulness", "truthfulness", "honesty"]
)
elif subset in ["flan"]:
principle = random.choice(
["helpfulness", "helpfulness", "helpfulness", "helpfulness", "verbalized_calibration"]
)
elif subset in ["evol_instruct"]:
principle = "helpfulness"
elif subset in ["truthful_qa", "false_qa"]:
principle = random.choice(["honesty", "truthfulness"])
# 10% chance of switching honesty to verbalized_calibration
if principle == "honesty":
principle = "honesty" if np.random.rand() < 0.9 else "verbalized_calibration"
# Select a specific prompt variant
principle_prompt = random.choice(principles[principle])