Environment:Haosulab ManiSkill Real Robot LeRobot Deps
| Knowledge Sources | |
|---|---|
| Domains | Real_Robot, Sim2Real |
| Last Updated | 2026-02-15 08:00 GMT |
Overview
Optional environment with HuggingFace LeRobot library for real robot control, camera interfaces, and motor bus communication in sim2real deployment.
Description
This environment provides the optional dependencies required for deploying trained policies to physical robots using the LeRobot framework. LeRobot provides hardware abstraction for cameras (`Camera`), motor buses (`MotorNormMode`), and robot coordination (`Robot`). ManiSkill's `LeRobotRealAgent` bridges the ManiSkill agent interface with LeRobot's hardware control, enabling direct policy deployment from simulation to real hardware.
Usage
Use this environment for the Sim2Real Deployment workflow when deploying to physical robots that are supported by the LeRobot framework (e.g., Koch, SO100 arms). This is required only for real-world deployment, not for simulation-only workflows.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux (Ubuntu 20.04+) | Real robot control typically requires Linux for hardware drivers |
| Hardware | USB ports for motor controllers | Required for physical robot communication |
| Hardware | USB cameras (optional) | For camera-based observation in real world |
| Python | >= 3.9 | Same as core environment |
Dependencies
System Packages
- USB serial drivers for motor controllers
- Camera drivers (depends on camera hardware)
Python Packages
- `lerobot` (from HuggingFace)
- All packages from Environment:Haosulab_ManiSkill_Python_SAPIEN_Core
Credentials
No additional credentials required. Physical robot communication uses USB serial ports.
Quick Install
# Install LeRobot
pip install lerobot
# Install ManiSkill
pip install mani_skill
Code Evidence
Optional LeRobot import from `mani_skill/agents/robots/lerobot/manipulator.py:15-21`:
try:
from lerobot.common.cameras.camera import Camera
from lerobot.common.motors.motors_bus import MotorNormMode
from lerobot.common.robots.robot import Robot
from lerobot.common.utils.robot_utils import busy_wait
except ImportError:
pass
Sim2Real CPU backend requirement from `mani_skill/envs/sim2real_env.py:66-68`:
assert (
self.sim_env.unwrapped.backend.sim_backend == "physx_cpu"
), "For the Sim2RealEnv we expect the simulation to be using the physx_cpu simulation backend"
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `ImportError` when using LeRobotRealAgent | LeRobot not installed | `pip install lerobot` |
| `AssertionError: ...physx_cpu simulation backend` | Using GPU backend with Sim2RealEnv | Set `sim_backend="cpu"` or `num_envs=1` for sim2real |
| USB permission denied | Missing serial port permissions | Add user to `dialout` group: `sudo usermod -a -G dialout $USER` |
Compatibility Notes
- Optional dependency: LeRobot is not required for simulation-only workflows. The import is wrapped in try/except and silently passes if unavailable.
- Sim2Real requires CPU backend: The `Sim2RealEnv` enforces `physx_cpu` simulation backend. GPU simulation cannot be used for real-world deployment.
- Supported robots: The LeRobot integration currently supports Koch and SO100 arms via dedicated `KochRealAgent` and `SO100RealAgent` classes.