Principle:FMInference FlexLLMGen Few Shot Prompt Construction
Metadata
| Field | Value |
|---|---|
| Sources | FlexLLMGen|https://github.com/FMInference/FlexLLMGen, Paper|GPT-3 Few-Shot|https://arxiv.org/abs/2005.14165 |
| Domains | Prompt_Engineering, NLP |
| Last updated | 2026-02-09 00:00 GMT |
Overview
A prompt engineering technique that constructs in-context learning prompts by prepending labeled examples before a query, enabling language models to perform classification and extraction tasks without fine-tuning.
Description
Few-shot prompting provides the model with k labeled examples (demonstrations) before the actual query. FlexLLMGen's data wrangling pipeline implements three strategies for selecting demonstrations:
- Manual - handcrafted examples from constants.PREFIXES specific to each dataset
- Random - randomly sampled from training data
- Validation/Hard - embedding-based selection using SentenceTransformers to find training examples most similar to the query (hard samples)
The prompt format concatenates serialized examples with their labels, separated by double newlines.
Usage
Use few-shot prompt construction when running LLM-based data wrangling tasks. Choose the strategy based on the task: manual for well-understood datasets, random for quick baselines, hard/validation for accuracy-critical tasks.
Theoretical Basis
In-context learning (Brown et al., 2020) enables LLMs to perform tasks by conditioning on demonstrations in the prompt. The choice and order of demonstrations significantly affects performance. Embedding-based selection (using cosine similarity of sentence embeddings) retrieves the most relevant examples for each query.