Implementation:Datajuicer Data juicer ImageWatermarkFilter

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Quality, Filtering
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for filtering data samples based on watermark detection probability in images provided by Data-Juicer.

Description

ImageWatermarkFilter is a filter operator that keeps samples whose images have no watermark with high probability. It uses a HuggingFace watermark detection model (default: amrul-hzz/watermark_detector) to predict the probability that each image contains a watermark. Samples are kept if the watermark probability is below the specified threshold. The key metric image_watermark_prob is cached in the stats field. The operator supports CUDA acceleration and 'any'/'all' strategies. It extends the Filter base class and implements the two-phase compute_stats/process pattern.

Usage

Import this operator when you need to filter out dataset samples containing watermarked images. Configure it in your Data-Juicer YAML config or instantiate directly.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/filter/image_watermark_filter.py
• Lines: 1-108

Signature

@OPERATORS.register_module("image_watermark_filter")
@LOADED_IMAGES.register_module("image_watermark_filter")
class ImageWatermarkFilter(Filter):
    def __init__(
        self,
        hf_watermark_model: str = "amrul-hzz/watermark_detector",
        trust_remote_code: bool = False,
        prob_threshold: float = 0.8,
        any_or_all: str = "any",
        *args,
        **kwargs,
    ):
        ...

Import

from data_juicer.ops.filter.image_watermark_filter import ImageWatermarkFilter

I/O Contract

Inputs

Name	Type	Required	Description
hf_watermark_model	str	No	Watermark detection model name on HuggingFace. Default: "amrul-hzz/watermark_detector"
trust_remote_code	bool	No	Whether to trust remote code of HF models. Default: False
prob_threshold	float	No	Watermark probability threshold (0 to 1). Samples with probability below this are kept. Default: 0.8
any_or_all	str	No	Keep strategy: 'any' or 'all' across images. Default: "any"

Outputs

Name	Type	Description
samples	Dict	Filtered samples with stats field updated (image_watermark_prob)

Usage Examples

YAML Configuration

process:
  - image_watermark_filter:
      hf_watermark_model: "amrul-hzz/watermark_detector"
      prob_threshold: 0.8
      any_or_all: "any"

Python API

from data_juicer.ops.filter.image_watermark_filter import ImageWatermarkFilter

op = ImageWatermarkFilter(prob_threshold=0.8)
# Apply to dataset
result = dataset.process(op)

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment