Implementation:NVIDIA NeMo Curator FineWebEduClassifier

Knowledge Sources	NVIDIA NeMo Curator
Domains	Text Classification, Quality Assessment, NLP
Last Updated	2026-02-14 00:00 GMT

Overview

The FineWebEduClassifier family implements three educational quality classifiers that score text on a 0-5 educational quality scale, optimized for multi-node, multi-GPU inference on large text datasets.

Description

This module provides three specialized classifiers for educational content assessment, all sharing a common architecture through the _FineWebBaseClassifier parent class:

FineWebModelStage extends ModelStage and wraps a HuggingFace AutoModelForSequenceClassification model. It configures a custom forward function that extracts logits and squeezes them. The process_model_output method converts raw logits into three output fields:

• Float score: Clamped to the range [0.0, 5.0]
• Integer score: Rounded and clamped to [0, 5]
• Label: "high_quality" if score >= 2.5, otherwise "low_quality"

_FineWebBaseClassifier is a CompositeStage[DocumentBatch, DocumentBatch] (dataclass) that chains together:

1. A TokenizerStage for text tokenization with DeBERTa padding configuration
2. A FineWebModelStage for model inference
3. An optional Filter stage for filtering by predicted category labels

The three concrete classifier classes differ only in their default HuggingFace model identifiers and output field naming conventions:

Classifier	Model ID	Default Batch Size
FineWebEduClassifier	HuggingFaceFW/fineweb-edu-classifier	256
FineWebMixtralEduClassifier	nvidia/nemocurator-fineweb-mixtral-edu-classifier	1024
FineWebNemotronEduClassifier	nvidia/nemocurator-fineweb-nemotron-4-edu-classifier	1024

The Mixtral variant was trained on the same text samples as the original FineWeb-Edu but using annotations from Mixtral 8x22B-Instruct. The Nemotron variant uses annotations from Nemotron-4-340B-Instruct.

All classifiers support autocast mode (enabled by default) which trades minor accuracy for faster inference, and input sorting by token length for improved batching performance.

Usage

Use FineWebEduClassifier and its variants when you need to assess the educational quality of text documents at scale. These classifiers are designed for quality filtering in data curation pipelines, enabling you to retain only high-quality educational content. The filter_by parameter allows automatic filtering by predicted labels (e.g., keep only "high_quality" documents).

Code Reference

Source Location

• Repository: NeMo-Curator
• File: nemo_curator/stages/text/classifiers/fineweb_edu.py
• Lines: 1-374

Signature

class FineWebEduClassifier(_FineWebBaseClassifier):
    def __init__(
        self,
        cache_dir: str | None = None,
        label_field: str = "fineweb-edu-score-label",
        float_score_field: str = "fineweb-edu-score-float",
        int_score_field: str = "fineweb-edu-score-int",
        text_field: str = "text",
        filter_by: list[str] | None = None,
        max_chars: int | None = None,
        sort_by_length: bool = True,
        model_inference_batch_size: int = 256,
        autocast: bool = True,
    ): ...

class FineWebMixtralEduClassifier(_FineWebBaseClassifier):
    def __init__(
        self,
        cache_dir: str | None = None,
        label_field: str = "fineweb-mixtral-edu-score-label",
        float_score_field: str = "fineweb-mixtral-edu-score-float",
        int_score_field: str = "fineweb-mixtral-edu-score-int",
        text_field: str = "text",
        filter_by: list[str] | None = None,
        max_chars: int | None = None,
        sort_by_length: bool = True,
        model_inference_batch_size: int = 1024,
        autocast: bool = True,
    ): ...

class FineWebNemotronEduClassifier(_FineWebBaseClassifier):
    def __init__(
        self,
        cache_dir: str | None = None,
        label_field: str = "fineweb-nemotron-edu-score-label",
        float_score_field: str = "fineweb-nemotron-edu-score-float",
        int_score_field: str = "fineweb-nemotron-edu-score-int",
        text_field: str = "text",
        filter_by: list[str] | None = None,
        max_chars: int | None = None,
        sort_by_length: bool = True,
        model_inference_batch_size: int = 1024,
        autocast: bool = True,
    ): ...

Import

from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebEduClassifier
from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebMixtralEduClassifier
from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebNemotronEduClassifier

I/O Contract

Inputs

Name	Type	Required	Description
cache_dir	None	No	HuggingFace cache directory for model weights
label_field	str	No	Name of the output prediction label column
float_score_field	str	No	Name of the output float score column
int_score_field	str	No	Name of the output integer score column
text_field	str	No (default: "text")	Name of the input text field in the data
filter_by	None	No	List of label values to keep (e.g., ["high_quality"]); None disables filtering
max_chars	None	No	Maximum character count for tokenizer input; None means no truncation
max_seq_length	int	No (default: 512)	Maximum sequence length for tokenization
sort_by_length	bool	No (default: True)	Whether to sort inputs by token length for efficient batching
model_inference_batch_size	int	No	Batch size for model inference
autocast	bool	No (default: True)	Enable autocast for faster inference with minor accuracy trade-off

Outputs

Name	Type	Description
label_field	str	Predicted label: "high_quality" (score >= 2.5) or "low_quality"
float_score_field	float	Float educational quality score clamped to [0.0, 5.0]
int_score_field	int	Integer educational quality score rounded and clamped to [0, 5]

Pipeline Architecture

The classifier internally decomposes into a multi-stage pipeline:

Input DocumentBatch
    |
    v
[TokenizerStage] -- Tokenizes text using DeBERTa tokenizer (right padding, max 512 tokens)
    |
    v
[FineWebModelStage] -- Runs AutoModelForSequenceClassification inference, produces scores and labels
    |
    v
[Filter (optional)] -- Filters documents by predicted label category
    |
    v
Output DocumentBatch (with added score/label columns)

Usage Examples

Basic Usage

from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebEduClassifier

# Create the classifier
classifier = FineWebEduClassifier(
    cache_dir="/path/to/hf_cache",
    text_field="text",
    model_inference_batch_size=256,
)

Filtering High-Quality Content

from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebEduClassifier

# Create classifier that only keeps high-quality documents
classifier = FineWebEduClassifier(
    filter_by=["high_quality"],
    max_chars=10000,
    sort_by_length=True,
)

Using Mixtral Variant

from nemo_curator.stages.text.classifiers.fineweb_edu import FineWebMixtralEduClassifier

# Use the Mixtral-annotated variant with larger batch size
classifier = FineWebMixtralEduClassifier(
    model_inference_batch_size=1024,
    autocast=True,
)

Related Pages

• Environment:NVIDIA_NeMo_Curator_Python_Linux_Base