Implementation:Huggingface Optimum TaskProcessorsManager

Knowledge Sources	Huggingface_Optimum
Domains	Preprocessing, Factory_Pattern
Last Updated	2026-02-15 00:00 GMT

Overview

Concrete tool for retrieving and instantiating task-specific dataset processors via a factory pattern provided by the Huggingface Optimum library.

Description

TaskProcessorsManager is a factory class that maps task names to their corresponding TaskProcessor subclasses. It supports four tasks: text-classification, token-classification, question-answering, and image-classification. The for_task class method instantiates the appropriate processor with provided arguments.

Usage

Use TaskProcessorsManager.for_task(task_name, config, preprocessor) to obtain a ready-to-use TaskProcessor instance without needing to import specific subclasses directly.

Code Reference

Source Location

• Repository: Huggingface_Optimum
• File: optimum/utils/preprocessing/task_processors_manager.py
• Lines: 1-50

Signature

class TaskProcessorsManager:
    _TASK_TO_DATASET_PROCESSING_CLASS = {
        "text-classification": TextClassificationProcessing,
        "token-classification": TokenClassificationProcessing,
        "question-answering": QuestionAnsweringProcessing,
        "image-classification": ImageClassificationProcessing,
    }

    @classmethod
    def get_task_processor_class_for_task(cls, task: str) -> Type["TaskProcessor"]:
        """Return the TaskProcessor class for the given task name."""

    @classmethod
    def for_task(cls, task: str, *args, **kwargs) -> "TaskProcessor":
        """Instantiate a TaskProcessor for the given task."""

Import

from optimum.utils.preprocessing.task_processors_manager import TaskProcessorsManager

I/O Contract

Inputs (for_task)

Name	Type	Required	Description
task	str	Yes	Task name (text-classification, token-classification, question-answering, image-classification)
*args	Any	Yes	Positional args passed to TaskProcessor.__init__ (config, preprocessor)
**kwargs	Any	No	Keyword args passed to TaskProcessor.__init__

Outputs

Name	Type	Description
processor	TaskProcessor	Instantiated task-specific processor

Usage Examples

from transformers import AutoConfig, AutoTokenizer
from optimum.utils.preprocessing.task_processors_manager import TaskProcessorsManager

config = AutoConfig.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# Get a text classification processor via the factory
processor = TaskProcessorsManager.for_task("text-classification", config, tokenizer)
dataset = processor.load_default_dataset(load_smallest_split=True)

# Get the class without instantiation
cls = TaskProcessorsManager.get_task_processor_class_for_task("question-answering")
print(cls.__name__)  # QuestionAnsweringProcessing

Related Pages

• Environment:Huggingface_Optimum_Python_Core_Dependencies