Implementation:Snorkel team Snorkel SliceAwareClassifier Score Slices

Knowledge Sources	Snorkel
Domains	Evaluation, Data_Slicing
Last Updated	2026-02-14 20:00 GMT

Overview

Concrete tool for evaluating per-slice model performance using the base prediction head, provided by the Snorkel library.

Description

The SliceAwareClassifier.score_slices() method evaluates the model on each slice by remapping slice-specific prediction labels to the base task. This ensures evaluation uses the master head (which combines all slice representations) rather than individual slice heads. Indicator labels are excluded from evaluation.

Usage

Call this method after training a SliceAwareClassifier to get per-slice performance metrics. Use as_dataframe=True for easy inspection.

Code Reference

Source Location

• Repository: snorkel
• File: snorkel/slicing/sliceaware_classifier.py
• Lines: L127-178

Signature

class SliceAwareClassifier(MultitaskClassifier):
    @torch.no_grad()
    def score_slices(
        self,
        dataloaders: List[DictDataLoader],
        as_dataframe: bool = False,
    ) -> Union[Dict[str, float], pd.DataFrame]:
        """
        Score using base task on each slice's prediction labels.

        Args:
            dataloaders: DictDataLoaders to evaluate.
            as_dataframe: Return as DataFrame (True) or dict (False).
        Returns:
            Per-slice metrics as dict ("task/dataset/split/metric" -> float)
            or DataFrame with columns [label, dataset, split, metric, score].
        """

Import

from snorkel.slicing import SliceAwareClassifier

I/O Contract

Inputs

Name	Type	Required	Description
dataloaders	List[DictDataLoader]	Yes	Dataloaders with slice labels
as_dataframe	bool	No	Return as DataFrame (default False)

Outputs

Name	Type	Description
metrics (dict)	Dict[str, float]	"task/dataset/split/metric" mapped to score
metrics (DataFrame)	pd.DataFrame	Columns: label, dataset, split, metric, score

Usage Examples

# After training
results = model.score_slices(
    dataloaders=[test_dl],
    as_dataframe=True,
)
print(results)
# Shows per-slice accuracy and F1 for each slice

Related Pages

Implements Principle

• Principle:Snorkel_team_Snorkel_Slice_Performance_Evaluation

Uses Heuristic

• Heuristic:Snorkel_team_Snorkel_Binary_Only_Slicing