Principle:ARISE Initiative Robomimic Dataset Download
| Knowledge Sources | |
|---|---|
| Domains | Robotics, Data_Pipeline, Data_Acquisition |
| Last Updated | 2026-02-15 08:00 GMT |
Overview
A registry-based dataset acquisition pattern that downloads pre-collected robot demonstration HDF5 files from remote repositories using configurable download sources and a centralized dataset registry.
Description
Dataset Download provides standardized access to benchmark demonstration datasets. Robot demonstration datasets are large HDF5 files containing recorded trajectories (states, actions, observations) from simulation environments. These files are hosted on HuggingFace Hub or Stanford servers and need to be downloaded before any data preparation or training can begin.
The principle uses a centralized DATASET_REGISTRY that maps (task, dataset_type, hdf5_type) tuples to download filenames and URLs. This registry is populated at package import time via register_all_links(), ensuring all available datasets are discoverable. The download mechanism supports two backends:
- HuggingFace Hub: For most benchmark datasets, using the hf_hub_download API
- Direct URL: For real-world datasets hosted on Stanford servers, using urllib
Usage
Use this principle as the first step in the Dataset Preparation Pipeline, before observation extraction or train/validation splitting. Users select a task (e.g., "lift", "can", "square"), dataset type (e.g., "ph" for proficient-human, "mh" for multi-human), and HDF5 type (e.g., "low_dim", "image").
Theoretical Basis
The dataset download principle implements a registry pattern with dual download backends:
# Abstract pattern (not real implementation)
DATASET_REGISTRY = {}
def register_dataset_link(task, dataset_type, hdf5_type, link):
key = (task, dataset_type, hdf5_type)
DATASET_REGISTRY[key] = link
# Usage: resolve a dataset key to a download action
link = DATASET_REGISTRY[(task, dataset_type, hdf5_type)]
if link.startswith("http"):
download_url(link, download_dir) # Direct URL download
else:
download_file_from_hf(repo_id, link, download_dir) # HuggingFace Hub