Ye Yuan

I'm a Senior Research Scientist in the Learning and Perception Team at NVIDIA Research. I received my Ph.D. in Robotics from Carnegie Mellon University in 2022, where I was advised by Prof. Kris Kitani. I also earned my M.S. in computer science at CMU in 2016, where I worked with Prof. Stelian Coros. I obtained my B.E. in computer science and technology from Zhejiang University in 2015. My research has been supported by the Qualcomm Innovation Fellowship and the NVIDIA Graduate Fellowship.

Email  |  CV  |  Google Scholar  |  Twitter  |  Github

profile photo
News
July 2023 Two papers accepted to CVPR 2024.
July 2023 Two papers accepted to ICCV 2023. See you in Paris!
Mar 2023 One paper on learning digital tennis player from videos accepted to SIGGRAPH 2023 (Best Paper Honorable Mention).
Feb 2023 One paper on simulating pedestrian motions accepted to CVPR 2023.
Sep 2022 One paper on embodied human pose estimation accepted to NeurIPS 2022.
May 2022 Joined NVIDIA Research as a Research Scientist.
Apr 2022 Defended my Ph.D. thesis Unified Simulation, Perception, and Generation of Human Behavior.
Mar 2022 One paper on global human mesh recovery accepted to CVPR 2022 with an Oral Presentation.
Jan 2022 One paper on efficient automatic agent design accepted to ICLR 2022 with an Oral Presentation.
Show more
Research

My research lies at the intersection of computer vision, machine learning, and robotics. My current focus is on digital humans, 3D generative AI, and embodied AI. Example topics: 3D human avatar generation, human motion generation, humanoid control, etc.

AGG: Amortized Generative 3D Gaussians for Single Image to 3D
Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, Arash Vahdat
arXiv, 2024
project page | arXiv | video
GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning
Ye Yuan*, Xueting Li*, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, Umar Iqbal   (*Equal Contribution)
CVPR, 2024   (Highlight)
project page | arXiv | video
PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
Jingbo Wang, Zhengyi Luo, Ye Yuan, Yixuan Li, Bo Dai
CVPR, 2024
project page | arXiv | video | code
PACE: Human and Camera Motion Estimation from in-the-wild Videos
Muhammed Kocabas, Ye Yuan, Pavlo Molchanov, Yunrong Guo, Michael Black, Otmar Hilliges, Jan Kautz, Umar Iqbal
3DV, 2024   (Spotlight Presentation)
project page | arXiv | video
PhysDiff: Physics-Guided Human Motion Diffusion Model
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz
ICCV, 2023   (Oral Presentation)
project page | arXiv | video
Learning Human Dynamics in Autonomous Driving Scenarios
Jingbo Wang, Ye Yuan, Zhengyi Luo, Kevin Xie, Dahua Lin, Umar Iqbal, Sanja Fidler, Sameh Khamis
ICCV, 2023
arXiv
Learning Physically Simulated Tennis Players from Broadcast Videos
Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Xue Bin Peng, Kayvon Fatahalian
SIGGRAPH, 2023   (Best Paper Honorable Mention)
project page | paper | video
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Sanja Fidler, Or Litany
CVPR, 2023
project page | arXiv | demo
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree, Charles Loop, Alexander Schwing, Stan Birchfield
ICRA, 2023
project page | arXiv | video | data
Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani
NeurIPS, 2022
project page | arXiv | video
Unified Simulation, Perception, and Generation of Human Behavior
Ye Yuan
Ph.D. Thesis, Robotics Institute, CMU, 2022
arXiv
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, Jan Kautz
CVPR, 2022   (Oral Presentation - Top 4.2%)
project page | arXiv | video | code
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris Kitani
ICLR, 2022   (Oral Presentation - Top 1.6%)
project page | arXiv | openreview | code
Online No-regret Model-Based Meta RL for Personalized Navigation
Yuda Song, Ye Yuan, Wen Sun, Kris Kitani
Learning for Dynamics & Control (L4DC), 2022
paper
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
Zhengyi Luo, Ryo Hachiuma, Ye Yuan, Kris Kitani
NeurIPS, 2021
project page | arXiv | video | code
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Ye Yuan, Xinshuo Weng, Yanglan Ou, Kris Kitani
ICCV, 2021
project page | arXiv | code
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih
CVPR, 2021   (Oral Presentation - Top 4.2%)
project page | arXiv | talk | video
PTP: Parallelized 3D Tracking and Prediction with Graph Neural Networks and Diversity Sampling
Xinshuo Weng*, Ye Yuan*, Kris Kitani   (*Equal Contribution)
RA-L and ICRA, 2021   (Best Student Paper Candidate < 2%)
project page | arXiv | code
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Ye Yuan, Kris Kitani
NeurIPS, 2020
project page | arXiv | video | code
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
Ye Yuan, Kris Kitani
ECCV, 2020
project page | arXiv | talk | summary | video | code
Efficient Non-Line-of-Sight Imaging from Transient Sinograms
Mariko Isogawa, Dorian Yao Chan, Ye Yuan, Kris Kitani, Matthew O'Toole
ECCV, 2020
project page | arXiv | summary | video
Diverse Trajectory Forecasting with Determinantal Point Processes
Ye Yuan, Kris Kitani
ICLR, 2020
arXiv | openreview | video
Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation
Mariko Isogawa, Ye Yuan, Matthew O'Toole, Kris Kitani
CVPR, 2020
project page | arXiv | video | code
Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
Jiaqi Guan, Ye Yuan, Kris Kitani, Nick Rhinehart
CVPR, 2020   (Oral Presentation - Top 5.7%)
arXiv | data
Back-Hand-Pose: 3D Hand Pose Estimation for a Wrist-worn Camera via Dorsum Deformation Network
Erwin Wu, Ye Yuan, Hui-Shyong Yeo, Aaron Quigley, Hideki Koike, Kris Kitani
ACM Symposium on User Interface Software and Technology (UIST), 2020
paper | video
MonoEye: Multimodal Human Motion Capture System Using A Single Ultra-Wide Fisheye Camera
Dong-Hyun Hwang, Kohei Aso, Ye Yuan, Kris Kitani, Hideki Koike
ACM Symposium on User Interface Software and Technology (UIST), 2020
paper | video
Ego-Pose Estimation and Forecasting as Real-Time PD Control
Ye Yuan, Kris Kitani
ICCV, 2019
project page | arXiv | video | code | data
3D Ego-Pose Estimation via Imitation Learning
Ye Yuan, Kris Kitani
ECCV, 2018
paper | video
Computational Design of Transformables
Ye Yuan, Changxi Zheng, Stelian Coros
ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), 2018
paper | video
Computational Abstractions for Interactive Design of Robotic Devices
Ruta Desai, Ye Yuan, Stelian Coros
ICRA, 2017
paper | video
Continuous Optimization of Interior Carving in 3D Fabrication
Yue Xie, Ye Yuan, Xiang Chen, Changxi Zheng, Kun Zhou
Frontiers of Computer Science, 2017
paper
Service
Conference Reviewer NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, AAAI, ICRA, SIGGRAPH, Eurographics
Journal Reviewer JMLR, TMLR, TPAMI, TIP, RA-L

Template adapted from this awesome website.