ABSTRACT:

Understanding how people speak when they interact with spoken dialogue systems is critical to improving the performance of those systems. In this talk, I will describe several experiments on data collected from subjects using TOOT, a system for accessing train information by phone, showing a) that user corrections of system errors differ from other user speech along several prosodic dimensions; and b) that prosodic information, together with other automatically available features, can be used to classify turns as corrections with a high degree of accuracy. I will discuss additional characteristics of user correction behavior that should be useful in designing future dialogue systems. For example, understanding whether speakers are more likely to repeat or rephrase their utterances, add new information or shorten their input, and how system behavior influences these choices, suggests new strategies for appropriate on-line modifications to a dialogue system's interaction strategy or to the recognition procedures it employs. I will describe plans to incorporate our findings into a dialogue system for our SCANMail project, which allows search and browsing of voicemail data by content. (This is joint work with Diane Litman (AT&T) and Marc Swerts (IPO).)