Implementation:Recommenders team Recommenders Download Utils

Knowledge Sources	Recommenders
Domains	Recommender Systems, Data Loading, File Management
Last Updated	2026-02-10 00:00 GMT

Overview

Provides core file download and extraction utilities used throughout the datasets subpackage for fetching remote datasets.

Description

The download_utils module contains three foundational functions for reliable file downloading and extraction. maybe_download checks if a file already exists locally before downloading; if not, it streams the file from a URL with a tqdm progress bar and optional expected byte size verification. It is decorated with @retry for automatic retry with random backoff (1-5 seconds, up to 5 attempts) to handle transient network failures. If a filename is not specified, it is extracted from the URL. After download, it optionally verifies the file size and removes the file if verification fails. download_path is a context manager that yields either a user-specified path (resolved to its real path) or a temporary directory that is automatically cleaned up on exit via TemporaryDirectory. unzip_file extracts all files from a zip archive to a destination directory with optional cleanup of the source zip file.

Usage

Use maybe_download whenever you need to download a dataset file from a URL with caching and retry support. Use download_path as a context manager to manage temporary download directories. Use unzip_file to extract downloaded zip archives. These functions are used internally by nearly all dataset loaders in the library (MovieLens, MIND, Criteo, Amazon Reviews, etc.).

Code Reference

Source Location

• Repository: Recommenders
• File: recommenders/datasets/download_utils.py
• Lines: 1-103

Signature

@retry(wait_random_min=1000, wait_random_max=5000, stop_max_attempt_number=5)
def maybe_download(
    url,
    filename=None,
    work_directory=".",
    expected_bytes=None,
) -> str

@contextmanager
def download_path(path=None) -> str

def unzip_file(zip_src, dst_dir, clean_zip_file=False) -> None

Import

from recommenders.datasets.download_utils import (
    maybe_download,
    download_path,
    unzip_file,
)

I/O Contract

Inputs

Name	Type	Required	Description
url	str	Yes (for maybe_download)	URL of the file to download.
filename	str or None	No (default: None)	Target filename. If None, extracted from the URL.
work_directory	str	No (default: ".")	Directory to save the downloaded file.
expected_bytes	int or None	No (default: None)	Expected file size in bytes for verification. File is removed if size does not match.
path	str or None	No (default: None)	Path for download_path context manager. If None, a temporary directory is used.
zip_src	str	Yes (for unzip_file)	Path to the zip file to extract.
dst_dir	str	Yes (for unzip_file)	Destination directory for extraction.
clean_zip_file	bool	No (default: False)	Whether to delete the zip file after extraction.

Outputs

Name	Type	Description
return (maybe_download)	str	File path of the downloaded (or already existing) file.
return (download_path)	str	Real path to the download directory (yielded as context manager).
return (unzip_file)	None	Files are extracted to dst_dir as a side effect.

Usage Examples

Basic Usage

from recommenders.datasets.download_utils import (
    maybe_download,
    download_path,
    unzip_file,
)

# Download a file with caching (skips if already exists)
filepath = maybe_download(
    url="https://example.com/dataset.zip",
    work_directory="/tmp/data",
)

# Use a temporary download directory that auto-cleans
with download_path() as path:
    filepath = maybe_download(
        url="https://example.com/dataset.zip",
        work_directory=path,
    )
    # Process the file within this block
    unzip_file(filepath, path, clean_zip_file=True)
# Temporary directory is cleaned up here

# Use a persistent directory
with download_path("/data/cache") as path:
    filepath = maybe_download(
        url="https://example.com/dataset.zip",
        work_directory=path,
    )

Dependencies

• requests - HTTP streaming downloads
• tqdm - Progress bar for download display
• retrying - Automatic retry with random backoff
• os / zipfile / tempfile - File system operations and archive extraction
• logging / math - Logging and byte calculation

Related Pages

Requires Environment

• Environment:Recommenders_team_Recommenders_Python_Core_Dependencies