Events

PhD Defense Conditional Generative Modeling for Holistic Human Behaviors
Export Event Save iCal Send to Google Calendar Send to Yahoo Calendar Send to Outlook Live Send to Microsoft Outlook Save iCal Preserve formatting in description (only supported in some calendar applications) Close Friday, March 21, 2025, 11:00am - 01:00pm

Speaker: Che-jui Chang

Bio

Location : CoRE 301

Committee: 

Professor Mubbasir Kapadia

Professor Vladimir Pavlovic

Professor Dimitris Metaxas

Professor Funda Durupinar (External)

Event Type: PhD Defense

Abstract: Animating behaviors for virtual humans and digital characters has immense potential for creating immersive and entertaining metaverse experiences in computer graphics and virtual reality. However, it remains challenging to craft lifelike human behaviors because natural human performance is inherently a synchronization of multiple modalities, and while existing approaches can generate individual aspects of human behavior, they often fail to capture the nuanced interplay between emotions, movements, and social interactions. Our approach to addressing these challenges is conditional generative modeling, which aims to synthesize human behaviors with high fidelity, achieve synchronization, and provide controllability for the performance.The first part of this dissertation advances the understanding of conversational behavior generation through novel emotional modeling. We begin by identifying critical limitations in existing embodied conversational agents, particularly their inability to maintain affect consistency across different behavioral modalities. Based on these findings, we propose an emotion-conditioned generative model that successfully disentangles content and emotion from input speech, enabling the generation of emotionally coherent facial animations. Our framework demonstrates significant improvements in both the naturalness and emotional expressiveness of virtual human performances compared to previous methods. The second part extends our conditioning approach to broader human movements and social interactions. We introduce CASIM, a semantic injection mechanism that fundamentally reimagines text-to-motion generation by incorporating the composite nature of human motions in both spatial and temporal domains. Our extensive experiments on the HumanML3D and KIT benchmarks demonstrate substantial improvements in text-motion correspondence and motion quality across multiple state-of-the-art models. Furthermore, we pioneer the generation of group activities by developing a novel diffusion-based framework with an interaction transformer that models inter-person dynamics, capable of synthesizing socially interacting groups of arbitrary sizes. This includes creating the first comprehensive dataset and evaluation metrics for group activity generation, establishing new benchmarks for this emerging research direction.

Organization: 

Contact  Professor Mubbasir Kapadia

Zoom link:
https://rutgers.zoom.us/j/97670996781?pwd=8xz5rKaIeXWCyJbaPAdyPkI7cyK5Wh.1