Implementation:Huggingface Datasets Interleave Datasets

Knowledge Sources	Huggingface Datasets HF Datasets Docs
Domains	Data_Engineering, NLP
Last Updated	2026-02-14 18:00 GMT

Overview

Concrete tool for alternating between multiple datasets with optional sampling probabilities provided by the HuggingFace Datasets library.

Description

interleave_datasets is a function that takes a list of Dataset or IterableDataset objects and returns a single interleaved dataset. When probabilities is None (the default), the function cycles through the sources in round-robin order. When probabilities are provided, examples are drawn from a random source at each step according to the given distribution, using the specified seed for reproducibility. Three stopping strategies are available: "first_exhausted" (stop when any source runs out, the default), "all_exhausted" (oversample until every source has been fully consumed at least once), and "all_exhausted_without_replacement" (each sample appears at most once). The function works with both map-style and iterable datasets.

Usage

Use interleave_datasets when training on a mixture of datasets and you want to control the sampling ratio between sources. This is common in multilingual training, multi-task learning, and domain adaptation.

Code Reference

Source Location

• Repository: datasets
• File: src/datasets/combine.py
• Lines: L18-L165

Signature

def interleave_datasets(
    datasets: list[DatasetType],
    probabilities: Optional[list[float]] = None,
    seed: Optional[int] = None,
    info: Optional[DatasetInfo] = None,
    split: Optional[NamedSplit] = None,
    stopping_strategy: Literal["first_exhausted", "all_exhausted", "all_exhausted_without_replacement"] = "first_exhausted",
) -> DatasetType:

Import

from datasets import interleave_datasets

I/O Contract

Inputs

Name	Type	Required	Description
datasets	list[Dataset] or list[IterableDataset]	Yes	List of datasets to interleave. All must be the same type.
probabilities	Optional[list[float]]	No	Sampling probabilities for each source. Must sum to 1.0 if provided. None uses round-robin.
seed	Optional[int]	No	Random seed for reproducible sampling when probabilities are provided.
info	Optional[DatasetInfo]	No	Dataset information to assign to the result.
split	Optional[NamedSplit]	No	Name of the dataset split to assign to the result.
stopping_strategy	str	No	When to stop: "first_exhausted" (default), "all_exhausted", or "all_exhausted_without_replacement".

Outputs

Name	Type	Description
dataset	Dataset or IterableDataset	The interleaved dataset. Type matches the input datasets.

Usage Examples

Basic Usage

from datasets import Dataset, interleave_datasets

d1 = Dataset.from_dict({"a": [0, 1, 2]})
d2 = Dataset.from_dict({"a": [10, 11, 12]})
d3 = Dataset.from_dict({"a": [20, 21, 22]})

# Round-robin interleaving
dataset = interleave_datasets([d1, d2, d3])
print(dataset["a"])  # [0, 10, 20, 1, 11, 21, 2, 12, 22]

# Probabilistic sampling
dataset = interleave_datasets(
    [d1, d2, d3],
    probabilities=[0.7, 0.2, 0.1],
    seed=42,
)
print(dataset["a"])  # [10, 0, 11, 1, 2]

# Oversampling: continue until all sources exhausted
dataset = interleave_datasets(
    [d1, d2, d3],
    probabilities=[0.7, 0.2, 0.1],
    seed=42,
    stopping_strategy="all_exhausted",
)

Related Pages

Implements Principle

• Principle:Huggingface_Datasets_Dataset_Interleaving