CS Events

PhD Defense

Towards Synergy Between Reinforcement Learning and Generative Models

 

Download as iCal file

Thursday, September 12, 2024, 08:00pm - 09:30pm

 

Speaker: Fei Deng

Location : CoRE 301

Committee

Prof. Sungjin Ahn (Chair)

Prof. Vladimir Pavlovic

Prof. Hao Wang

Prof. Kimin Lee (External)

Event Type: PhD Defense

Abstract: Recent advances in reinforcement learning (RL) and generative models have led to groundbreaking achievements. However, both fields encounter significant challenges: RL methods often exhibit high sample complexity, while generative models frequently face misalignment with human preferences. This dissertation addresses these challenges by exploring the synergistic relationship between RL and generative models, and is structured into two main parts. The first part focuses on model-based reinforcement learning (MBRL), which enhances data efficiency for RL by incorporating a generative world model that can simulate the environment. We tackle two primary limitations of the leading MBRL agent, Dreamer. First, to address the short-range memory constraints of Dreamer's RNN-based world model, we introduce S4WM, the first world model framework that integrates the long-range memory capabilities of state space models. Second, to mitigate Dreamer's sensitivity to visual distractions caused by its reconstruction-based training, we develop a novel reconstruction-free MBRL agent called DreamerPro. The second part explores the potential of RL to improve generative models. We propose PRDP, a novel RL method for aligning diffusion models with human preferences. The stable large-scale training enabled by PRDP significantly enhances the quality of generated images on complex and unseen prompts.

Contact  Professor Sungjin Ahn

Zoom Link: https://rutgers.zoom.us/j/99581176829?pwd=BDs4psE1mf0PlTQpRDxDH5r0HT1NoY.1