DexJoCo¶
cleanrua_utils/dexjoco_env.py
wraps the task environments of DexJoCo, a MuJoCo benchmark for dexterous
manipulation, as a gym.Env. Each arm is a Franka Panda with a 16-joint Allegro hand. Scenes, control and success
checks are DexJoCo's own.
from cleanrua_utils.dexjoco_env import DexJoCoEnv
env = DexJoCoEnv('/path/to/dexjoco', task='pick_bucket')
observation, info = env.reset(seed=0) # info['instruction'] is the task instruction.
Spaces¶
Action: one 50 Hz control step, ten physics substeps of DexJoCo's operational-space controller.
{'arms': {arm: {'pose': (7,), # target flange [x, y, z, qw, qx, qy, qz]
'hand': (16,)}}} # Allegro joint targets in radians
Observation:
{'arms': {arm: {'pose': (7,), # measured flange pose
'hand': (16,), # measured hand joints
'hand_command': (16,)}}, # last commanded hand joints
'cameras': {name: {'rgb': (640, 640, 3),
'intrinsic_K': (3, 3),
'T_world_from_optical': (4, 4)}}, # x right, y down, z forward
'native_steps': int}
Arms are right for single-arm tasks and right, left for bimanual ones. Hand joints are ordered index, middle,
ring and thumb, four each. The cameras are those the multi-task Pi0.5 policy was trained with, for example front
and wrist for pick_bucket. Reward is 1 on task success.
Methods¶
| Method | What it does |
|---|---|
native_observation() |
Images and the 46-dimensional state in the format of DexJoCo's multi-task Pi0.5 policy |
evaluate() |
The official verdict: success, and score 100 or 0, since DexJoCo has no partial credit |
Tasks¶
Single arm: pick_bucket, click_mouse, fold_glasses, hammer_nail, pinch_tongs, water_plant. Bimanual:
bimanual_assembly, bimanual_hanoi, bimanual_microwave_cook, bimanual_photograph, bimanual_unlock_ipad.
Instructions and cameras come from configs/multi_task/<task>.yaml; step limits range from 1,000 to 1,500 control
steps. Any seed works, and the same seed gives the same scene.
Practical notes¶
- Images render with EGL unless
MUJOCO_GLis set. - DexJoCo paces its steps to 30 Hz of wall-clock time; the wrapper turns that off, so episodes run as fast as the simulator allows.
- For
click_mousethe wrapper first holds the recorded start pose for 30 steps, as DexJoCo's evaluation does.