Implementation:Datajuicer Data juicer DetectCharacterLocationsMapper

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Processing, Mapping
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for detecting and extracting bounding boxes for named main characters in images provided by Data-Juicer.

Description

DetectCharacterLocationsMapper is a mapper operator that detects and extracts bounding boxes for main characters in an image using a combination of YOLOE object detection, multimodal language model captioning (LLaVA), and BLIP image-text matching. It uses a YOLOE model to detect candidate object regions, then generates captions for each region using a multimodal LLM. It matches cropped image regions against character names using a BLIP image-text matching filter, applying IoU and matching score thresholds to resolve overlapping detections. Final bounding boxes are stored in the metadata under main_character_locations_list. Requires CUDA acceleration. It extends the Mapper base class.

Usage

Import when you need to spatially localize named characters in images for character analysis pipelines.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/mapper/detect_character_locations_mapper.py

Signature

@OPERATORS.register_module("detect_character_locations_mapper")
class DetectCharacterLocationsMapper(Mapper):
    def __init__(self,
                 mllm_mapper_args: Optional[Dict] = {},
                 image_text_matching_filter_args: Optional[Dict] = {},
                 yoloe_path="yoloe-11l-seg.pt",
                 iou_threshold=0.7,
                 matching_score_threshold=0.4,
                 *args, **kwargs):

Import

from data_juicer.ops.mapper.detect_character_locations_mapper import DetectCharacterLocationsMapper

I/O Contract

Inputs

Name	Type	Required	Description
mllm_mapper_args	Optional[Dict]	No	Arguments for the multimodal language model mapper controlling caption generation. Default empty dict uses fixed values (max_new_tokens=256, temperature=0.2, hf_model=llava-hf/llava-v1.6-vicuna-7b-hf)
image_text_matching_filter_args	Optional[Dict]	No	Arguments for the image-text matching filter. Default empty dict uses fixed values (min_score=0.1, max_score=1.0, hf_blip=Salesforce/blip-itm-base-coco)
yoloe_path	str	No	Path to the YOLOE model weights. Default: "yoloe-11l-seg.pt"
iou_threshold	float	No	IoU threshold for considering two bounding boxes as overlapping. Default: 0.7
matching_score_threshold	float	No	Matching score threshold for character-image matching. Default: 0.4

Outputs

Name	Type	Description
samples	Dict	Transformed samples with main_character_locations_list added to metadata containing bounding boxes per character

Usage Examples

YAML Configuration

process:
  - detect_character_locations_mapper:
      yoloe_path: yoloe-11l-seg.pt
      iou_threshold: 0.7
      matching_score_threshold: 0.4

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment