CS Events

PhD Defense

Disentangled Generative Models and their Applications

 

Download as iCal file

Tuesday, March 29, 2022, 01:00pm - 03:00pm
 

Speaker: Bingchen Liu

Location : Via Zoom

Committee: 

Ahmed Elgammal (elgammal@cs.rutgers.edu)

Vladimir Pavlovic (vladimir@cs.rutgers.edu)

Yongfeng Zhang (yongfeng.zhang@rutgers.edu)

Xiaohui Shen (shenxiaohui@bytedance.com).

Event Type: PhD Defense

Abstract: Generative models are fascinating for their ability to synthesize versatile visual, audio, and textual information from mere noise. This generative process usually requires a model to perceive and compress high-dimensional data into a compact, low-dimensional latent space, where each dimension encodes valuable semantic variations in the original data space. Disentangling generative models makes them more fun to play with. This thesis studies the unsupervised disentangling of the latent space in GANs focused on the image domain and further extended to multi-modalities (image captioning and text-to-image synthesis). Derived from disentanglement, this thesis also covers studies on model interpretability and human-controllable data synthesis. First, we work on general-purpose unsupervised disentanglement. A novel GAN-based disentanglement framework with One-Hot Sampling and Orthogonal Regularization (OOGAN) is proposed. Second, we tackle a more specific task: disentangling coarse and fine level style attributes for GAN. We design a Vector-Quantized module for better pose-identity disentanglement and a novel joint-training scheme merging GAN and Auto-Encoder. Lastly, we study two applications taking advantage of a better disentangled GAN with mutual information learning, which are text-and-image mutual-translation and sketch-to-image generation.

Organization: 

Rutgers University School of Arts and Sciences

Contact  Ahmed Elgammal

zoom link: https://rutgers.zoom.us/j/97137433843?pwd=U01GZ1VIYnA0VXlZVlNXZkw1Y3F2Zz09#success,

meeting ID: 97137433843 Passcode: 360210