Skip to content Skip to navigation
PhD Defense
9/30/2013 10:00 am
CBIM Multipurpose Room ( Room 22 )

Facial Expression Transfer by using 3D-Aware Expression Flow

Fei Yang, Rutgers University

Defense Committee: Dimitris Metaxas (chair), Tina Eliassi-Rad, Kostas Bekris and Xiaolei Huang(Lehigh University)

Abstract

We address the problem of editing facial expression in images and videos, such as replacing local facial components, or exaggerating a facial expression. Direct copying and blending facial components results in semantically unnatural composites, since expression is a global effect and the local face component in one expression is often incompatible with the shape and other components in another expression. To solve this problem we present the expression flow method, which is a 2D flow field that can warp the target face globally. We develop a tensor-based 3D face geometry reconstruction method, which fits a 3D model for each input image, with the constraint that all models have the same identity. With the identity constraint, the differences between the underlying 3D shapes capture only changes in expression and pose. We apply our algorithms in a series of applications, and show that our system is able to generate faces with much higher fidelity than existing methods.