Implementation:Datajuicer Data juicer LLMQualityScoreFilter
| Knowledge Sources | |
|---|---|
| Domains | Data_Quality, Filtering |
| Last Updated | 2026-02-14 16:00 GMT |
Overview
Concrete tool for filtering data samples based on quality scores estimated by a language model provided by Data-Juicer.
Description
LLMQualityScoreFilter is a filter operator that keeps samples with a high quality score estimated by a language model. It uses an LLM to evaluate the quality of each sample across multiple dimensions: accuracy, grammar, informativeness, and coherence. Each dimension is scored on a 1-5 scale. The overall quality score is the average of these dimensions. The key metrics llm_quality_score and llm_quality_record are cached in the stats field. It extends LLMAnalysisFilter and implements the two-phase compute_stats/process pattern.
Usage
Import this operator when you need to filter dataset samples based on text quality as assessed by a language model. Configure it in your Data-Juicer YAML config or instantiate directly.
Code Reference
Source Location
- Repository: Datajuicer_Data_juicer
- File: data_juicer/ops/filter/llm_quality_score_filter.py
- Lines: 1-98
Signature
@OPERATORS.register_module("llm_quality_score_filter")
class LLMQualityScoreFilter(LLMAnalysisFilter):
# Inherits __init__ from LLMAnalysisFilter
# DEFAULT_SYSTEM_PROMPT provides evaluation rubric
# DEFAULT_DIM_REQUIRED_KEYS = ["accuracy", "grammar", "informativeness", "coherence"]
...
Import
from data_juicer.ops.filter.llm_quality_score_filter import LLMQualityScoreFilter
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| api_or_hf_model | str | No | API or HuggingFace model name. Default: "gpt-4o" (inherited from LLMAnalysisFilter) |
| min_score | float | No | The minimum quality score to keep samples. Default: 0.5 (inherited) |
| is_hf_model | bool | No | Indicates if the model is from HuggingFace. Default: False (inherited) |
Outputs
| Name | Type | Description |
|---|---|---|
| samples | Dict | Filtered samples with stats field updated (llm_quality_score, llm_quality_record) |
Usage Examples
YAML Configuration
process:
- llm_quality_score_filter:
min_score: 0.5
max_score: 1.0
Python API
from data_juicer.ops.filter.llm_quality_score_filter import LLMQualityScoreFilter
op = LLMQualityScoreFilter(min_score=0.5, max_score=1.0)
# Apply to dataset
result = dataset.process(op)