Publications

  • SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
  • Zhengyi Luo†, Ye Yuan†, Tingwu Wang†, Chenran Li†, Fernando Castañeda†, Sirui Chen*, Zi-Ang Cao*, Jiefeng Li*, David Minor*, Qingwei Ben*, Jinhyung Park*, David Sami*, Zi Wang*, Xingye Da*, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi "Jim" Fan, Yuke Zhu
  • In Submission
  • † Co-First Authors, * Core Contributors
  • [Paper] [Project]
  • VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
  • Tairan He*, Zi Wang*, Haoru Xue*, Qingwei Ben*, Zhengyi Luo, Wenli Xiao, Ye Yuan, Xingye Da, Fernando Castañeda, Shankar Sastry, Changliu Liu, Guanya Shi, Linxi "Jim" Fan, Yuke Zhu
  • CVPR 2026
  • * Equal contribution
  • [Paper] [Project]
  • Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
  • Haoru Xue*, Tairan He*, Zi Wang*, Qingwei Ben, Wenli Xiao, Zhengyi Luo, Ye Yuan, Xingye Da, Fernando Castañeda, Guanya Shi, Shankar Sastry, Linxi "Jim" Fan, Yuke Zhu
  • CVPR 2026
  • * Equal contribution
  • [Paper] [Project]
  • BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
  • Yitang Li*, Zhengyi Luo*, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta, Guanya Shi
  • ICLR 2026
  • * Equal contribution
  • [Paper] [Project]
  • Self-Improving Vision-Language-Action Models with Data Generation via Residual RL
  • Wenli Xiao*, Haotian Lin*, Andy Peng, Haoru Xue, Tairan He, Yuqi Xie, Fengyuan Hu, Jimmy Wu, Zhengyi Luo, Linxi Fan, Guanya Shi, Yuke Zhu
  • ICLR 2026
  • * Equal contribution
  • [Paper] [Project]
  • HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos
  • Haoyang Weng, Yitang Li, Nikhil Sobanbabu, Zihan Wang, Zhengyi Luo, Tairan He, Deva Ramanan, Guanya Shi
  • In Submission
  • [Paper] [Project] [Code]
  • Learning Universal Humanoid Control
  • Zhengyi Luo
  • PhD Thesis
  • [Paper]
  • MaskedManipulator: Versatile Whole-Body Control for Loco-Manipulation
  • Chen Tessler, Yifeng Jiang, Erwin Coumans, Zhengyi Luo, Gal Chechik, Xue Bin Peng
  • SIGGRAPH Asia 2025
  • [Paper] [Code]
  • UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Control
  • Yan Wu, Korrawe Karunratanakul, Zhengyi Luo, Siyu Tang
  • ICCV 2025 (Highlight)
  • [Paper] [Project] [Code]
  • Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning
  • Zhengyi Luo*, Chen Tessler*, Toru Lin, Ye Yuan, Tairan He, Wenli Xiao, Yunrong Guo, Gal Chechik, Kris Kitani, Linxi "Jim" Fan, Yuke Zhu
  • 1st Workshop on Humanoid Agents, CVPR 2025 (Best Paper Award)
  • * Equal contribution
  • [Paper] [Project]
  • ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills
  • Tairan He*, Jiawei Gao*, Wenli Xiao*, Yuanhang Zhang*, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica Hodgins, Linxi Fan, Yuke Zhu
  • RSS 2025
  • [Paper] [Project] [Code]
  • ISMimic: Learning Basketball Interaction Skills from Demonstrations
  • Yinhuai Wang*, Qihan Zhao*, Runyi Yu*, Ailing Zeng, Jing Lin, Zhengyi Luo, Hok Wai Tsui, Jiwen Yu, Xiu Li, Qifeng Chen, Jian Zhang, Lei Zhang, Ping Tan
  • CVPR 2025
  • * Equal contribution
  • [Paper] [Project] [Code]
  • HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
  • Tairan He*, Wenli Xiao*, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu
  • ICRA 2025
  • * Equal contribution
  • [Paper] [Project] [Code]
  • CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
  • Guy Tevet, Sigal Raab, Setareh Cohan, Daniele Reda, Zhengyi Luo, Xue Bin Peng, Amit H. Bermano, Michiel van de Panne
  • ICLR 2025
  • [Paper] [Project] [Code]
  • SMPLOlympics: Sports Environments for Physically Simulated Humanoids
  • Zhengyi Luo*, Jiashun Wang*, Kangni Liu*, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, Jessica Hodgins, Kris Kitani
  • 1st Workshop on Humanoid Agents, CVPR 2025
  • * Equal contribution
  • [Paper] [Project] [Code]
  • Harmony4d: A video dataset for in-the-wild close human interactions
  • Rawal Khirodkar*, Jyun-Ting Song*, Jinkun Cao, Zhengyi Luo, Kris Kitani
  • NeurIPS Datasets and Benchmarks 2024
  • [Paper] [Project] [Code]
  • Grasping Diverse Objects with Simulated Humanoids
  • Zhengyi Luo*, Jinkun Cao*, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu
  • NeurIPS 2024
  • * Equal contribution
  • [Paper] [Project]
  • OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
  • Tairan He*, Zhengyi Luo*, Xialin He*, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, Guanya Shi
  • CORL 2024
  • * Equal contribution
  • [Paper] [Project]
  • Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
  • Tairan He*, Zhengyi Luo*, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, Guanya Shi
  • IROS 2024 (Oral Presentation)
  • * Equal contribution
  • [Paper] [Project]
  • Real-Time Simulated Avatar from Head-Mounted Sensors
  • Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu
  • CVPR 2024 (Highlight — Top 11%)
  • [Paper] [Project] [Code]
  • PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
  • Jingbo Wang*, Zhengyi Luo*, Ye Yuan, Yixuan Li, Bo Dai
  • CVPR 2024
  • * Equal contribution
  • [Paper]
  • Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
  • K Grauman et al.
  • CVPR 2024 (Oral — Top 3%)
  • [Paper] [Project]
  • PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction
  • Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang, Lei Zhang
  • In Submission
  • [Paper] [Project]
  • Universal Humanoid Motion Representations for Physics-Based Control
  • Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu
  • ICLR 2024 (Spotlight — Top 5%)
  • [Paper] [Project] [Code]
  • Perpetual Humanoid Control for Real-time Simulated Avatars
  • Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu
  • ICCV 2023
  • [Video] [Paper] [Project] [Code]
  • Learning Human Dynamics in Autonomous Driving Scenarios
  • Jingbo Wang, Ye Yuan, Zhengyi Luo, Kevin Xie, Dahua Lin, Umar Iqbal, Sanja Fidler, Sameh Khamis
  • ICCV 2023
  • [Paper]
  • Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
  • Davis Rempe*, Zhengyi Luo*, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, Or Litany
  • CVPR 2023
  • * Equal contribution
  • [Video] [Paper] [Project]
  • Embodied Scene-aware Human Pose Estimation
  • Zhengyi Luo*, Shun Iwase*, Ye Yuan, Kris M. Kitani
  • NeurIPS 2022
  • * Equal contribution
  • [Video] [Paper] [Project] [Code]
  • From Universal Humanoid Control to Automatic Physically Valid Character Creation
  • Zhengyi Luo, Ye Yuan, Kris M. Kitani
  • ArXiv
  • [Paper] [Project] [Code]
  • Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
  • Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris M. Kitani
  • ICLR 2022 (Oral Presentation — Top 1.6%)
  • [Paper] [Project] [Code]
  • Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
  • Zhengyi Luo, Ryo hachiuma, Ye Yuan, Kris M. Kitani
  • ICCVW (CV4ARVR) 2021, NeurIPS 2021
  • [Video] [Paper] [Project] [Code]
  • 3D Human Motion Estimation via Motion Compression and Refinement
  • Zhengyi Luo, S. Alireza Golestaneh, Kris M. Kitani
  • ACCV 2020
  • Oral Presentation
  • [Video] [Paper] [Project] [Code]
  • Learning Shape Representations for Clothing Variations in Person Re-Identification
  • Yu-Jhe Li, Zhengyi Luo, Xinshuo Weng, Kris M. Kitani
  • arXiv, 2020
  • [Paper]
  • Cross-Domain 3D Equivariant Image Embeddings
  • Carlos Esteves, Avneesh Sud, Zhengyi Luo, Kostas Daniilidis, Ameesh Makadia
  • ICML 2019
  • [Paper]
  • Cloud Chaser: Real Time Deep Learning Computer Vision on Low Computing Power Devices
  • Zhengyi Luo, Austin Small, Liam Dugan, Stephen Lane
  • ICMV 2018
  • [Paper] [Project] [Code]
  • The rural–urban stress divide: Obtaining geographical insights through Twitter
  • Kokil Jaidka, Sharath Chandra Guntuku, Jane H Lee, Zhengyi Luo, Anneke Buffone, Lyle H Ungar
  • Computers in Human Behavior, 2020: 106544
  • [Paper] [Project]

---