Implementation:Datajuicer Data juicer QueryTopicDetectionMapper

Knowledge Sources	Datajuicer_Data_juicer
Domains	Data_Processing, Mapping
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for detecting topic labels in user queries provided by Data-Juicer.

Description

QueryTopicDetectionMapper is a mapper operator that predicts the topic label and confidence score for user queries using a Hugging Face topic classification model. It optionally translates Chinese queries to English before classification. The predicted topic label and score are stored in the sample metadata under configurable keys. Samples that already have topic annotations are skipped. Requires CUDA acceleration and operates in batched mode.

Usage

Use when you need automatic topic-based annotation of training data queries for topic-aware data curation and analysis workflows.

Code Reference

Source Location

• Repository: Datajuicer_Data_juicer
• File: data_juicer/ops/mapper/query_topic_detection_mapper.py

Signature

@OPERATORS.register_module("query_topic_detection_mapper")
class QueryTopicDetectionMapper(Mapper):
    def __init__(
        self,
        hf_model: str = "dstefa/roberta-base_topic_classification_nyt_news",
        zh_to_en_hf_model: Optional[str] = "Helsinki-NLP/opus-mt-zh-en",
        model_params: Dict = {},
        zh_to_en_model_params: Dict = {},
        *,
        label_key: str = MetaKeys.query_topic_label,
        score_key: str = MetaKeys.query_topic_score,
        **kwargs,
    ):

Import

from data_juicer.ops.mapper.query_topic_detection_mapper import QueryTopicDetectionMapper

I/O Contract

Inputs

Name	Type	Required	Description
hf_model	str	No	HuggingFace model ID for topic classification (default: dstefa/roberta-base_topic_classification_nyt_news)
zh_to_en_hf_model	Optional[str]	No	Translation model from Chinese to English (default: Helsinki-NLP/opus-mt-zh-en)
model_params	Dict	No	Model parameters for the topic classification model
zh_to_en_model_params	Dict	No	Model parameters for the translation model
label_key	str	No	Key name in meta field for the output label (default: query_topic_label)
score_key	str	No	Key name in meta field for the output score (default: query_topic_score)

Outputs

Name	Type	Description
meta[label_key]	str	Predicted topic label
meta[score_key]	float	Confidence score for the predicted topic

Usage Examples

process:
  - query_topic_detection_mapper:
      hf_model: 'dstefa/roberta-base_topic_classification_nyt_news'
      zh_to_en_hf_model: 'Helsinki-NLP/opus-mt-zh-en'

Related Pages

• Environment:Datajuicer_Data_juicer_Python_Runtime_Environment