Speaking with Hands:
Creating Animated Conversational Characters from
Recordings of Human Performance
"That was ugly, dude! On this run you forgot to set up your jump."
Abstract:
We describe a method for using a database of recorded speech
and captured motion to create an animated conversational character.
People's utterances are composed of short, clearly-delimited
phrases; in each phrase, gesture and speech go together
meaningfully and synchronize at a common point of maximum emphasis.
We develop tools for collecting and managing performance data that
exploit this structure. The tools help create scripts for
performers, help annotate and segment performance data, and
structure specific messages for characters to use within
application contexts. Our animations then reproduce this
structure. They recombine motion samples with new speech samples
to recreate coherent phrases, and blend segments of speech and
motion together phrase-by-phrase into extended utterances. By
framing problems for utterance generation and synthesis so that
they can draw closely on a talented performance, our techniques
support the rapid construction of animated characters with rich and
appropriate expression.
Publications:
Matthew Stone, Doug DeCarlo, Insuk Oh, Christian Rodriguez,
Adrian Stere, Alyssa Lees, Chris Bregler. Speaking with Hands: Creating Animated Conversational
Characters from Recordings of Human Performance
In SIGGRAPH 2004.
