Implementation:Snorkel team Snorkel PandasTFApplier Apply

Knowledge Sources	Snorkel
Domains	Data_Augmentation, Data_Pipeline
Last Updated	2026-02-14 20:00 GMT

Overview

Concrete tool for applying transformation functions to a Pandas DataFrame using a configured policy to produce augmented data, provided by the Snorkel library.

Description

The PandasTFApplier class applies TFs to a DataFrame according to a policy. It provides two modes:

• apply(): Returns the complete augmented DataFrame (batch mode)
• apply_generator(): Yields augmented DataFrames in batches (memory-efficient generator)

Both modes iterate over data points, sample TF sequences from the policy, apply them, and collect successful transformations.

Usage

Import this class when you have defined TFs and a policy and need to generate augmented data.

Code Reference

Source Location

• Repository: snorkel
• File: snorkel/augmentation/apply/pandas.py
• Lines: L9-65

Signature

class PandasTFApplier(TFApplier):
    def apply(
        self,
        df: pd.DataFrame,
        progress_bar: bool = True,
    ) -> pd.DataFrame:
        """
        Apply TFs to DataFrame.

        Args:
            df: Input DataFrame.
            progress_bar: Show tqdm progress.
        Returns:
            Augmented DataFrame with original + transformed rows.
        """

    def apply_generator(
        self,
        df: pd.DataFrame,
        batch_size: int,
    ) -> pd.DataFrame:
        """
        Generator that yields augmented DataFrames in batches.

        Args:
            df: Input DataFrame.
            batch_size: Rows per yielded batch.
        Yields:
            Augmented DataFrame batches.
        """

Import

from snorkel.augmentation import PandasTFApplier

I/O Contract

Inputs

Name	Type	Required	Description
tfs	List[TransformationFunction]	Yes	TFs (passed to constructor)
policy	Policy	Yes	Augmentation policy (passed to constructor)
df	pd.DataFrame	Yes	Input data to augment
progress_bar	bool	No	Show progress (default True)

Outputs

Name	Type	Description
apply() result	pd.DataFrame	Augmented DataFrame (originals + transformed rows)
apply_generator() result	Generator[pd.DataFrame]	Batched augmented DataFrames

Usage Examples

import pandas as pd
from snorkel.augmentation import PandasTFApplier, RandomPolicy

# Assume tfs = [tf_replace_word, tf_add_prefix] are defined
policy = RandomPolicy(n_tfs=2, n_per_original=2, keep_original=True)

applier = PandasTFApplier(tfs=tfs, policy=policy)
df = pd.DataFrame({"text": ["hello world", "good morning"], "label": [0, 1]})

# Batch mode
df_augmented = applier.apply(df)
print(f"Original: {len(df)}, Augmented: {len(df_augmented)}")

# Generator mode for large datasets
for batch in applier.apply_generator(df, batch_size=100):
    process(batch)

Related Pages

Implements Principle

• Principle:Snorkel_team_Snorkel_Transformation_Application