George Jiayuan Gao

I am a second year Robotics Master’s student in the General Robotics, Automation, Sensing & Perception (GRASP) Laboratory at the University of Pennsylvania, advised by Prof. Nadia Figueroa and Prof. Dinesh Jayaraman.

Previously, I completed my undergraduate studies in Computer Science and Mathematics at Washington University in St. Louis, where I worked with Prof. Yevgeniy Vorobeychik.

My research focuses on the intersection of learning-based methods and control theory for robotics, with the goal of enabling robots to safely and intelligently interact with the physical world.

News

Nov 09, 2024	Presented our work on Object-Centric Recovery at CoRL 2024 Workshop on Lifelong Learning for Home Robots. Presentation Video.
Oct 26, 2024	Our paper “Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy For Visuomotor Imitation Learning” was accepted for Spotlight at the CoRL 2024 Workshop on Lifelong Learning for Home Robots.

Publications

Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy For Visuomotor Imitation Learning

George Jiayuan Gao, Tianyu Li, and Nadia Figueroa

Spotlight at CoRL Workshop on Lifelong Learning for Home Robots, 2024.

Abs arXiv PDF Video Project Website

We propose an object-centric recovery policy framework to address the challenges of out-of-distribution (OOD) scenarios in visuomotor policy learning. Previous behavior cloning (BC) methods rely heavily on a large amount of labeled data coverage, failing in unfamiliar spatial states. Without relying on extra data collection, our approach learns a recovery policy constructed by an inverse policy inferred from object keypoint manifold gradient in the original training data. The recovery policy serves as a simple add-on to any base visuomotor BC policy, agnostic to a specific method, guiding the system back towards the training distribution to ensure task success even in OOD situations. We demonstrate the effectiveness of our object-centric framework in both simulation and real robot experiments, achieving an improvement of 77.7% over the base policy in OOD.

Selected Projects

(On-Going) Robust Visuomotor Behavior Cloning Via Novel View Synthesis and Stable Action Synthesis

Proposed pipeline for novel consistent view and stable action generation from single robot demonstration as data-augmentation scheme to enable significantly more data-efficient visuomotor policy learning, 2024.

Page
(On-Going) Learning Globally-Latently-Asymptotically Stable Visuomotor Policy

Can vision-based robot policy output actions that converges to some latent attractor? Is it better to separate the learning of latent states with the learning of latent dynamics? We aim to investigate this question in this project, 2024.

Page
Novel Environment Transfer of Visuomotor Policy Via Object-Centric Domain-Randomization

Proposed GDN-ACT, a novel, scalable approach that enables zero-shot generalization of visuomotor policies across unseen environments, using a pre-trained state-space mapping for object localization, May 2024.

PDF
Modular Gait Optimization: From Unit Moves to Multi-Step Trajectory in Bipedal Systems

Proposed the Gait Modularization and Optimization Technique (GMOT), which leverages modular unit gaits as initialization for Hybrid Direct Collocation (HDC), reducing sensitivity to constraints and enhancing computational stability across various gaits, including walking, running, and hopping, Dec 2023.

PDF Code
Miniature City Autonomous Driving Platform Development with Real-Time Vision-Based Lane-Following

Developed the drive stack for Washington University’s inaugural miniature city autonomous driving platform by developing the vision-based lane-following pipeline, May 2023.

Code