Events

PhD Defense Deep Generative Models for Long-Horizon Decision-Making
Export Event Save iCal Send to Google Calendar Send to Yahoo Calendar Send to Outlook Live Send to Microsoft Outlook Save iCal Preserve formatting in description (only supported in some calendar applications) Close Thursday, April 10, 2025, 09:00pm - 11:00pm

Speaker: Chang Chen

Bio

Location : Virtual

Committee: 

Professor Sungjin Ahn

Professor Ruixiang Tang

Professor Yongfeng Zhang

Professor Amy Zhang (external)

Event Type: PhD Defense

Abstract:  Model-based deep reinforcement learning (MBRL) leverages learned models of environment dynamics to facilitate more efficient planning and decision-making. Recent advancements have improved model accuracy, uncertainty estimation, and planning in latent spaces, enabling MBRL to perform competitively in complex control tasks. However, when applied to long-horizon problems, MBRL still faces critical challenges, including compounding model errors, difficulties in credit assignment under sparse rewards, and limited generalization beyond the training distribution. This dissertation addresses these challenges through innovations in both architectural design and modeling strategies, and is organized into three main parts. In the first part, we show that employing Transformer-based world models significantly improves long-horizon prediction accuracy by effectively capturing long-range temporal dependencies. However, the use of autoregressive generation introduces compounding errors over time. In the second part, we reformulate reinforcement learning as a sequence modeling problem and demonstrate that a hierarchical diffusion planner can achieve strong performance in long-horizon planning tasks. To address the limitations of diffusion planners in accurately estimating value, we further show that integrating value learning at the lower level enables the hierarchical diffusion planner to excel in both sparse-reward and dense-reward environments. Finally, in the third part, we tackle a fundamental limitation of sequence modeling approaches: their inability to generalize beyond the training distribution. As a distribution modeling technique, the diffusion planner is constrained by the data it has seen and struggles to generate valid plans when start-goal connections are absent. To overcome this, we propose a stitch-and-plan framework that enables generalization beyond the training data by composing complete trajectories from feasible sub-trajectories, effectively stitching together partial plans to form coherent solutions.

Organization: 

Contact  Professor Sungjin Ahn

https://rutgers.zoom.us/j/92638959851?pwd=YFxabDSFIW4DoPjmas9k2j0C0EHmSb.1