Implementation:Datajuicer Data juicer ImageTextMatchingFilter

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Quality, Filtering
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for filtering data samples based on image-text matching scores provided by Data-Juicer.

Description

ImageTextMatchingFilter is a filter operator that keeps samples with image-text matching scores within a specific range. It uses a HuggingFace BLIP model (default: Salesforce/blip-itm-base-coco) to compute matching scores between images and text. The scores can be reduced using 'avg', 'max', or 'min' modes when one text corresponds to multiple images. The operator supports horizontal and vertical flipping of images before computation. The key metric image_text_matching_score is cached in the stats field. It supports CUDA acceleration and 'any'/'all' strategies. It extends the Filter base class and implements the two-phase compute_stats/process pattern.

Usage

Import this operator when you need to filter dataset samples based on how well images match their associated text. Configure it in your Data-Juicer YAML config or instantiate directly.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/filter/image_text_matching_filter.py
• Lines: 1-165

Signature

@OPERATORS.register_module("image_text_matching_filter")
@LOADED_IMAGES.register_module("image_text_matching_filter")
class ImageTextMatchingFilter(Filter):
    def __init__(
        self,
        hf_blip: str = "Salesforce/blip-itm-base-coco",
        trust_remote_code: bool = False,
        min_score: float = 0.003,
        max_score: float = 1.0,
        horizontal_flip: bool = False,
        vertical_flip: bool = False,
        any_or_all: str = "any",
        reduce_mode: str = "avg",
        *args,
        **kwargs,
    ):
        ...

Import

from data_juicer.ops.filter.image_text_matching_filter import ImageTextMatchingFilter

I/O Contract

Inputs

Name	Type	Required	Description
hf_blip	str	No	BLIP model name on HuggingFace for image-text matching. Default: "Salesforce/blip-itm-base-coco"
trust_remote_code	bool	No	Whether to trust remote code of HF models. Default: False
min_score	float	No	The minimum matching score to keep samples. Default: 0.003
max_score	float	No	The maximum matching score to keep samples. Default: 1.0
horizontal_flip	bool	No	Flip image horizontally (left to right). Default: False
vertical_flip	bool	No	Flip image vertically (top to bottom). Default: False
any_or_all	str	No	Keep strategy: 'any' or 'all' across images. Default: "any"
reduce_mode	str	No	Reduce mode for multiple images per chunk: 'avg', 'max', or 'min'. Default: "avg"

Outputs

Name	Type	Description
samples	Dict	Filtered samples with stats field updated (image_text_matching_score)

Usage Examples

YAML Configuration

process:
  - image_text_matching_filter:
      hf_blip: "Salesforce/blip-itm-base-coco"
      min_score: 0.003
      max_score: 1.0
      any_or_all: "any"
      reduce_mode: "avg"

Python API

from data_juicer.ops.filter.image_text_matching_filter import ImageTextMatchingFilter

op = ImageTextMatchingFilter(min_score=0.003, max_score=1.0)
# Apply to dataset
result = dataset.process(op)

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment