CS Events

Qualifying Exam Towards an emotionally expressive embodied conversational agent
Export Event Save iCal Send to Google Calendar Send to Yahoo Calendar Send to Outlook Live Send to Microsoft Outlook Save iCal Preserve formatting in description (only supported in some calendar applications) Close Monday, December 18, 2023, 01:00pm - 02:00pm

Speaker: Che-jui Chang

Location : CoRE 305

Committee: 

Associate Professor Mubbasir Kapadia (Chair)

Professor Vladimir Pavlovic

Distinguished Professor Dimitris Metaxas

Assistant Professor He Zhu

Event Type: Qualifying Exam

Abstract: Recent success in generative AI and large language models (LLMs) has led to a wide range of interesting applications, notably interactive virtual assistant and embodied conversational agents (ECAs).These applications are crucial in advancing human-computer interaction by providing immersive and expressive dialogues across various domains and enhancing user engagement.Constructing an ECA with immersive realism requires synchronous multi-modal responses as well as the integration of emotionally expressive behaviors in voice, facial expressions and body gestures.However, most existing works often entail a simple concatenation of modules such as text-to-speech and speech-to face and gesture, resulting in expressionless responses primarily because of the dilution in emotions across the modalities.In this talk, I plan to delve into this challenge and propose solutions through three distinctive research works. The first work focuses on refining the synchronization of audio and facial expressions with conditioned emotions. The second involves synthesizing body gestures that align with audio and textual inputs, ensuring semantic appropriateness. The last one highlights the importance of emotion conditioning and affect consistency for multimodal ECAs via large-scale user studies.All three combined provide insights for the development of expressive embodied conversational agents.

: 

Contact  Associate Professor Mubbasir Kapadia (Chair)