PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulation

Yifan Yin^*, Zhengtao Han^*, Shivam Aarya, Shuhang Xu, Jianxin Wang, Jiawei Peng, Angtian Wang, Alan Yuille, Tianmin Shu (^*Equal contribution) (WRL@ICLR 2025)

RSS, 2025

We developed a benchmark for part-level robot manipulation, training and evaluating various baselines at large-scale, including Diffusion Policy, 3D Diffusion Policy, Octo and Act3D. We explored the effect of part-level observation for policy learning, conducting ablation studies using part-level mask as an additional observation.