Implementation:Datajuicer Data juicer ImageNSFWFilter

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Quality, Filtering
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for filtering data samples based on NSFW scores of images provided by Data-Juicer.

Description

ImageNSFWFilter is a filter operator that keeps samples whose images have NSFW scores in a specified range. It uses a HuggingFace model (default: Falconsai/nsfw_image_detection) to compute NSFW scores for each image. The operator supports CUDA acceleration and 'any' (keep if any image meets the condition) or 'all' (keep only if all images meet the condition) strategies. The NSFW scores are cached under the image_nsfw_score stats key. It extends the Filter base class and implements the two-phase compute_stats/process pattern.

Usage

Import this operator when you need to filter dataset samples based on the NSFW content probability of images. Configure it in your Data-Juicer YAML config or instantiate directly.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/filter/image_nsfw_filter.py
• Lines: 1-109

Signature

@OPERATORS.register_module("image_nsfw_filter")
@LOADED_IMAGES.register_module("image_nsfw_filter")
class ImageNSFWFilter(Filter):
    def __init__(
        self,
        hf_nsfw_model: str = "Falconsai/nsfw_image_detection",
        trust_remote_code: bool = False,
        min_score: float = 0.0,
        max_score: float = 0.5,
        any_or_all: str = "any",
        *args,
        **kwargs,
    ):
        ...

Import

from data_juicer.ops.filter.image_nsfw_filter import ImageNSFWFilter

I/O Contract

Inputs

Name	Type	Required	Description
hf_nsfw_model	str	No	NSFW detection model name on HuggingFace. Default: "Falconsai/nsfw_image_detection"
trust_remote_code	bool	No	Whether to trust remote code of HF models. Default: False
min_score	float	No	The minimum NSFW score threshold (range 0 to 1). Default: 0.0
max_score	float	No	The maximum NSFW score threshold (range 0 to 1). Default: 0.5
any_or_all	str	No	Keep strategy: 'any' or 'all' across images. Default: "any"

Outputs

Name	Type	Description
samples	Dict	Filtered samples with stats field updated (image_nsfw_score)

Usage Examples

YAML Configuration

process:
  - image_nsfw_filter:
      hf_nsfw_model: "Falconsai/nsfw_image_detection"
      min_score: 0.0
      max_score: 0.5
      any_or_all: "any"

Python API

from data_juicer.ops.filter.image_nsfw_filter import ImageNSFWFilter

op = ImageNSFWFilter(min_score=0.0, max_score=0.5)
# Apply to dataset
result = dataset.process(op)

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment