HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots

[00:00] Introduction to Hover: Neural Whole Body Controller for Humanoids [00:15] Problem: Current controllers lack versatility across tasks [00:50] Human motion imitation as a unified control approach [01:23] Policy distillation: Learning from an oracle policy [02:01] Command space: Kinematic, joint angle, and root tracking modes [02:34] Motion retargeting: From human data to robot movements [03:09] Performance comparison with specialist policies [03:43] Real-world testing on Unitree H1 robot [04:15] Comparison with MHC and Masked Mimic approaches [04:49] Future work and current limitations [05:18] Reward function design and components [06:02] D-Agger advantages in policy learning [06:33] Domain randomization for sim-to-real transfer [07:06] Conclusions on Hover's contributions Authors: Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu Affiliations: NVIDIA, CMU, UC Berkeley, UT Austin, UC San Diego Abstract: Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications. Link: https://hover-versatile-humanoid.github.io/