Principle:Huggingface Open r1 Difficulty Based Data Filtering
Metadata
| Field | Value |
|---|---|
| Sources | Paper (Scaling LLM Test-Time Compute https://arxiv.org/abs/2408.03314), Blog (Open-R1 Blog https://huggingface.co/blog/open-r1) |
| Domains | NLP, Data_Engineering |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A data curation methodology that filters training problems by model-estimated difficulty using pass rate computation, retaining only problems within an optimal difficulty range for reinforcement learning training.
Description
Not all training problems are equally useful for RL training. Problems that are too easy (model always solves them) provide no learning signal, while problems that are too hard (model never solves them) also waste training compute. This principle addresses problem selection by:
- Generating multiple completions per problem using the training model itself.
- Scoring each completion with the same reward functions used during training.
- Computing the mean pass rate (fraction of correct solutions).
- Filtering to keep only problems where
pass_rate_min < mean_reward < pass_rate_max.
Typical thresholds are 0.1-0.9, keeping problems the model can sometimes but not always solve. This is related to curriculum learning and the "zone of proximal development" concept.
Usage
Use before GRPO training to curate the training dataset, ensuring problems are at the right difficulty level for the current model. Re-run when switching to a different base model.
Theoretical Basis
Pass Rate Filtering Logic
The core filtering algorithm generates multiple completions per problem, scores them with the training reward functions, and retains only problems where the mean reward falls within the specified bounds:
# Pseudocode: Pass rate filtering
for each problem in dataset:
completions = model.generate(problem, n=num_generations)
rewards = [reward_func(completion) for completion in completions]
pass_rate = mean(rewards, ignore_none=True)
if pass_rate_min < pass_rate < pass_rate_max:
keep(problem)
else:
discard(problem)