Implementation:Datajuicer Data juicer SpecifiedFieldFilter

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Quality, Filtering
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for filtering data samples based on specified field values provided by Data-Juicer.

Description

SpecifiedFieldFilter is a filter operator that keeps samples where a specified field's value matches any value in a target list. It extends Filter and uses the two-phase compute_stats/process pattern. In compute_stats_single, it traverses the sample dictionary using a dot-separated field_key to extract the field value and copies it to stats. In process_single, it checks whether all values in the field are present in target_value. Supports multi-level dot-separated field keys (e.g., level1.level2) and reversed_range to invert filter logic.

Usage

Import when filtering based on categorical field values. Configure in YAML or Python.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/filter/specified_field_filter.py

Signature

@OPERATORS.register_module("specified_field_filter")
class SpecifiedFieldFilter(Filter):
    def __init__(self, field_key: str = "", target_value: list = [], *args, **kwargs):

Import

from data_juicer.ops.filter.specified_field_filter import SpecifiedFieldFilter

I/O Contract

Inputs

Name	Type	Required	Description
field_key	str	No	Dot-separated key path to the target field (default: "")
target_value	list	No	List of acceptable values for the field (default: [])

Outputs

Name	Type	Description
samples	Dict	Filtered samples based on field value matching

Usage Examples

YAML Configuration

process:
  - specified_field_filter:
      field_key: "meta.source"
      target_value: ["wikipedia", "arxiv"]

Python API

from data_juicer.ops.filter.specified_field_filter import SpecifiedFieldFilter
op = SpecifiedFieldFilter(field_key="meta.source", target_value=["wikipedia", "arxiv"])

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment