Disentangled Generative Models and their Applications
Tuesday, March 29, 2022, 01:00pm - 03:00pm
Speaker: Bingchen Liu
Location : Via Zoom
Ahmed Elgammal (email@example.com)
Vladimir Pavlovic (firstname.lastname@example.org)
Yongfeng Zhang (email@example.com)
Xiaohui Shen (firstname.lastname@example.org).
Event Type: PhD Defense
Abstract: Generative models are fascinating for their ability to synthesize versatile visual, audio, and textual information from mere noise. This generative process usually requires a model to perceive and compress high-dimensional data into a compact, low-dimensional latent space, where each dimension encodes valuable semantic variations in the original data space. Disentangling generative models makes them more fun to play with. This thesis studies the unsupervised disentangling of the latent space in GANs focused on the image domain and further extended to multi-modalities (image captioning and text-to-image synthesis). Derived from disentanglement, this thesis also covers studies on model interpretability and human-controllable data synthesis. First, we work on general-purpose unsupervised disentanglement. A novel GAN-based disentanglement framework with One-Hot Sampling and Orthogonal Regularization (OOGAN) is proposed. Second, we tackle a more specific task: disentangling coarse and fine level style attributes for GAN. We design a Vector-Quantized module for better pose-identity disentanglement and a novel joint-training scheme merging GAN and Auto-Encoder. Lastly, we study two applications taking advantage of a better disentangled GAN with mutual information learning, which are text-and-image mutual-translation and sketch-to-image generation.
Rutgers University School of Arts and Sciences
Contact Ahmed Elgammal