Environment:ARISE Initiative Robomimic HuggingFace Hub Dependencies
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Data_Management, NLP |
| Last Updated | 2026-02-15 07:30 GMT |
Overview
HuggingFace Hub integration for dataset downloads and CLIP language embeddings, with pinned versions of `huggingface_hub`, `transformers`, and `diffusers`.
Description
Robomimic uses HuggingFace Hub for two purposes: (1) downloading benchmark datasets via `hf_hub_download` from the `robomimic/robomimic_datasets` repository, and (2) generating CLIP language embeddings for language-conditioned policies using `CLIPTextModelWithProjection` from the `openai/clip-vit-large-patch14` model. The `diffusers` library provides the DDPM/DDIM noise schedulers and EMA utilities for Diffusion Policy training. All three packages have pinned versions in `setup.py` to ensure reproducibility.
Usage
Use this environment when downloading robomimic datasets (via `download_datasets.py`) or when training language-conditioned policies that require CLIP text embeddings. The `diffusers` dependency is required specifically for the Diffusion Policy algorithm.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Mac OS X or Linux | Cross-platform via Python |
| Network | Internet access | Required for dataset downloads and model weight downloads |
| Disk | 5-50GB | Depends on number of datasets downloaded |
Dependencies
Python Packages
- `huggingface_hub` == 0.23.4 (pinned)
- `transformers` == 4.41.2 (pinned)
- `diffusers` == 0.11.1 (pinned)
Credentials
The following environment variables may be needed:
- `HF_HOME`: HuggingFace cache directory (defaults to `~/tmp`). CLIP model weights are cached at `$HF_HOME/clip/`.
- `TOKENIZERS_PARALLELISM`: Set to `"true"` automatically by robomimic to suppress deadlock warnings.
- `WANDB_ENTITY`: Weights & Biases entity name for experiment logging (set via `macros_private.py`).
- `WANDB_API_KEY`: Weights & Biases API key for experiment logging (set via `macros_private.py` or `wandb login`).
Quick Install
# All dependencies are installed with robomimic
pip install robomimic
# Or install individually (pinned versions)
pip install huggingface_hub==0.23.4 transformers==4.41.2 diffusers==0.11.1
Code Evidence
Pinned versions from `setup.py:32-34`:
"huggingface_hub==0.23.4",
"transformers==4.41.2",
"diffusers==0.11.1",
HuggingFace dataset download from `robomimic/utils/file_utils.py:555-580` (via `hf_hub_download`):
from huggingface_hub import hf_hub_download
CLIP language embedding from `robomimic/utils/lang_utils.py:1-10`:
from transformers import AutoModel, pipeline, AutoTokenizer, CLIPTextModelWithProjection
os.environ["TOKENIZERS_PARALLELISM"] = "true" # needed to suppress warning about potential deadlock
tokenizer = "openai/clip-vit-large-patch14"
lang_emb_model = CLIPTextModelWithProjection.from_pretrained(
tokenizer,
cache_dir=os.path.expanduser(os.path.join(os.environ.get("HF_HOME", "~/tmp"), "clip"))
).eval()
Diffusers usage from `robomimic/algo/diffusion_policy.py:12-15`:
# requires diffusers==0.11.1
from diffusers.schedulers.scheduling_ddpm import DDPMScheduler
from diffusers.schedulers.scheduling_ddim import DDIMScheduler
from diffusers.training_utils import EMAModel
W&B credential handling from `robomimic/utils/log_utils.py:63-68`:
if Macros.WANDB_API_KEY is not None:
os.environ["WANDB_API_KEY"] = Macros.WANDB_API_KEY
assert Macros.WANDB_ENTITY is not None, "WANDB_ENTITY macro is set to None."
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `ImportError: cannot import name 'DDPMScheduler'` | Wrong diffusers version | `pip install diffusers==0.11.1` |
| `AssertionError: WANDB_ENTITY macro is set to None` | W&B entity not configured | Run `python robomimic/scripts/setup_macros.py` and set WANDB_ENTITY in `macros_private.py` |
| `No private macro file found!` | `macros_private.py` not created | Run `python robomimic/scripts/setup_macros.py` |
| `ConnectionError` during dataset download | No internet or HuggingFace server unreachable | Check network; datasets can also be downloaded manually |
Compatibility Notes
- Version pinning: The `huggingface_hub`, `transformers`, and `diffusers` versions are strictly pinned. Using different versions may cause import errors or API incompatibilities.
- CLIP model caching: The CLIP model (`openai/clip-vit-large-patch14`) is downloaded on first use and cached at `$HF_HOME/clip/`. Set `HF_HOME` to control cache location.
- W&B logging: Optional. Configure via `macros_private.py` (created by `setup_macros.py`). Falls back to offline mode after 10 failed connection attempts.
- Real-world datasets: Some datasets (e.g., `lift_real`, `can_real`) are hosted on Stanford servers, not HuggingFace Hub.