Workflow:OpenBMB UltraFeedback Completion Generation
| Knowledge Sources | |
|---|---|
| Domains | LLMs, Inference, Data_Engineering |
| Last Updated | 2023-12-29 00:00 GMT |
Overview
End-to-end process for generating diverse LLM completions for a set of instructions by loading models, applying principle-guided system prompts, formatting architecture-specific prompts, and running inference via HuggingFace Transformers or vLLM.
Description
This workflow describes how a single instruction is transformed into a model-specific formatted prompt and then into a generated response. It covers the full completion generation sub-pipeline: loading a model from the 17-model pool, selecting a behavioral principle based on the instruction's source dataset, choosing a concrete system prompt from the principle's variant pool, formatting the prompt using architecture-specific conversation templates (LLaMA-2, Vicuna, Alpaca, MPT, Falcon, StarChat), and running inference with appropriate generation parameters. The pipeline supports two backends: HuggingFace Transformers (per-model sequential processing) and vLLM (batched high-throughput generation with tensor parallelism).
Usage
Execute this workflow when you need to generate LLM responses for a prepared instruction dataset. You need a JSON file of instructions with pre-assigned model lists (from the model sampling step), access to model checkpoints (local or HuggingFace Hub), and GPU resources. Use the HuggingFace backend for single-GPU setups or when working with commercial API models; use the vLLM backend for high-throughput batched generation across multiple GPUs.
Execution Steps
Step 1: Environment Setup
Install required dependencies and configure the runtime environment. The HuggingFace pipeline requires transformers, tokenizers, deepspeed, and accelerate at specific versions. The vLLM pipeline additionally requires the vLLM library and NCCL environment variables for multi-GPU communication. The launcher scripts handle dependency installation automatically.
Key considerations:
- HuggingFace backend pins transformers==4.31.0 and tokenizers==0.13.3
- vLLM backend requires NCCL_IGNORE_DISABLED_P2P=1 and RAY_memory_monitor_refresh_ms=0
- CUDA_LAUNCH_BLOCKING=1 is set for vLLM debugging
- TOKENIZERS_PARALLELISM=false prevents tokenizer warnings in multi-process settings
Step 2: Model Loading
Load the target model into GPU memory using the appropriate backend. For the HuggingFace backend, LLaMA-family models are loaded via LlamaForCausalLM.from_pretrained with automatic device mapping, while non-LLaMA models (MPT, Falcon, StarChat) use the generic pipeline interface with trust_remote_code. For vLLM, models are loaded as an LLM instance with configurable GPU memory utilization (95%), swap space, and automatic tensor parallelism across available GPUs. Commercial models (GPT-4, GPT-3.5) use the OpenAI API instead of local loading.
Key considerations:
- StarChat uses bfloat16 precision explicitly
- vLLM auto-detects dtype except for StarChat/MPT/Falcon which use bfloat16
- Model paths are mapped from short names (e.g., "ultralm-13b") to HuggingFace identifiers
- GPU memory utilization is set to 95% in vLLM for maximum throughput
Step 3: Principle and Prompt Assignment
For each instruction, determine the behavioral principle based on the source dataset and sample a concrete system prompt. The principle selection follows dataset-specific probability distributions (e.g., Evol-Instruct always uses helpfulness, TruthfulQA uses honesty/truthfulness). An additional rule converts 10% of honesty selections to verbalized calibration. The selected system prompt is then injected into the model's conversation template.
What happens:
- A principle category is sampled per dataset-specific distribution
- A concrete prompt string is randomly chosen from ~11 variants per principle
- The prompt is stored alongside the completion metadata for provenance tracking
Step 4: Prompt Formatting
Format the instruction and system prompt into the target model's expected input format using the fastchat conversation template library. Each model architecture has a distinct template: LLaMA-2 uses [INST]/[/INST] tokens with <<SYS>> blocks, Vicuna uses "USER:"/"ASSISTANT:" colon-separated turns, Alpaca uses "### Instruction"/"### Response" headers, MPT uses ChatML-style im_start/im_end tokens, Falcon uses RWKV-style formatting, StarChat uses pipe-delimited system/user/assistant tags, and UltraLM uses a custom "User:"/"Assistant:" format.
Key considerations:
- The Conversation dataclass manages prompt construction via get_prompt()
- System prompts are appended to the template's default system message
- Each template defines its own separator style, stop strings, and stop token IDs
- WizardLM-7B uses a simplified format without conversation templates
Step 5: Inference Execution
Run model inference to generate the completion text. The HuggingFace backend uses the transformers pipeline with temperature=1.0, top_p=1.0, max_new_tokens=1024, and do_sample=True, plus model-specific stopping criteria. The vLLM backend uses SamplingParams with equivalent settings and processes all prompts in a single batched call. Commercial models are called via the OpenAI ChatCompletion API with retry logic (up to 20 attempts). Generated text is post-processed by stripping whitespace and truncating at quadruple newlines.
Key considerations:
- Both backends use identical sampling parameters for consistency
- vLLM extracts stop strings from conversation templates for proper truncation
- The OpenAI API caller uses temperature=1 and max_tokens=1024
- Post-processing removes trailing EOS tokens and excessive whitespace
Step 6: Result Collection and Storage
Append each generated completion to the instruction's completions array, recording the model name, assigned principle, system prompt, and response text. Results are written back to the source JSON file, overwriting in-place. The HuggingFace backend processes data in shards (2000 instructions per run, controlled by an --id argument), while the vLLM backend processes the entire dataset at once.
Key considerations:
- The HuggingFace backend uses datasets.Dataset.map() for per-example processing
- The vLLM backend separates principle sampling (map) from generation (batched)
- Results are serialized as JSON with 4-space indentation
- Multiple model runs are executed sequentially, each appending to the same file