Knowledge Representation for Language

Matthew Stone

Statement of Current Research - Fall 2002


Overview
Statement
Pragmatics
Syntax
Semantics
Latest directions

Some Highlights
Computer Animation
Speech Prosody
New Logic Paper
Dialogue Management
SPUD
Teaching CL

Knowledge representation for language
My research is directed toward the ultimate goal of constructing conversational agents, programs that can understand and contribute to spoken dialogue with a human user in ordinary natural language. As yet, we struggle to imbue these agents with disarmingly simple capabilities, such as that of reference. To refer, a conversational agent must be able to offer a handful of words so its user can identify an object for use in an ongoing real-world task. These problems are challenging because they engage the perennial issues of artificial intelligence research: the uncertainty that infects everyday tasks; the complexity of the world; and the computational obstacles to putting information about the world to use. With reference, for example, we cannot avoid the common and useful case where the agent and its user are strangers, and neither has previously encountered the individual objects they must identify to one another, talk about and use.

My research in attacking such problems is guided by a number of general hypotheses about the use, structure and meaning of human language, which help to bridge the perspectives of artificial intelligence and the language sciences:

  • Language use is action; so, for example, language understanding is no more and no less than recognizing the intention behind another agent's actions.
  • Natural grammar describes language as action directly, and so organizes its alternatives and ambiguities to streamline language use.
  • Speakers' meanings fit their broader plans and intentions; thus, meaning connects closely with the ontology and information that interlocutors must maintain anyway, if they are perceive, deliberate and act together successfully in the world.
These hypotheses reflect a vision of language as a flexible and elegant system that allows intelligent agents to coordinate by drawing closely on their existing abilities to act in the world. In what follows I survey my research in computational pragmatics, syntax and semantics of natural language, and suggest how the convergence between linguistic theory and computation it represents promises steady progress not only towards models of people's competence in contributing to conversation but also towards practical implementations that reproduce it.

Computational pragmatics and planning
Pragmatics begins with the philosopher H. P. Grice's characterization of language use as intentional activity. I am working to develop this view by providing formal representations of collaborative intentions that directly accommodate concrete linguistic action;
my chapter in World-Situated Language Use (2002) offers an overview. What makes communicative intentions difficult to formalize is agents' changing and incomplete information: agents in collaborative conversations often coordinate abstractly on future decisions that cannot be made on present information (see the preliminary report in my AAAI Fall Symposium paper, 2001). Where AI research has typically required complex accounts of such choices characterized in terms of descriptions of actions and plans, I use inferences simply about what agents know (as in my AAAI paper, 1998). This substantial simplification rests on the structure of proofs in logics of knowledge. Reasoning about epistemic states tracks how available information could be used to compute decisions (formalized and proved in my paper in Theoretical Computer Science, 1999). Proof search in logics of knowledge can be implemented efficiently so as to respect this intuitive constraint on the use of information (proved in my paper in submission to ACM Transactions on Computational Logic). A number of my ongoing projects start from these results to explore and document the use of inferences about knowledge and action to implement conversational language use as collaborative reasoning.

Among its many advantages, this account of communicative intentions provides a uniform account of linguistic actions, nonlinguistic communicative actions, and many other kinds of deliberate, coordinated real-world action. It is thus particularly suited to multi-modal interactive communication (as in my collaboration with Justine Cassell that we report in International Natural Language Generation Conference (INLG), 2000).

Computational syntax and its interfaces with semantics and pragmatics
Computational approaches to grammar achieve particular efficiency and elegance when they are lexicalized---that is, when they package syntactic information directly with the words on which syntactic computations are performed. My research has explored lexicalized grammars with a particular focus on how they can be used for generation as well as understanding. Generation calls for richer representations of grammar than understanding does; syntactic forms must be linked to discourse context (as in
my collaboration with Christine Doran in the Association for Computational Linguistics (ACL) Conference 1997) and entries must be designed to analyze precisely the language of a particular corpus or application (as I and my colleagues have described in 2000's Tree-Adjoining Grammar Workshop). Generation also raises new problems in reasoning from the grammar; linguistic forms must be connected to the context (as in my own paper to INLG 2000), and to the effects they can achieve in conversation (as Bonnie Webber and I study in INLG 1998).

The SPUD paper (in submission to Computational Intelligence) integrates and extends this research. We adopt the perspective of lexicalized grammar to argue for a uniform approach to the problem of planning sentences, and to describe an implementation that instantiates it.

Computational semantics
Computational applications put the problem of the context-dependence of meaning into particular relief. The interpretation of utterances rests on general knowledge of meaning, but reveals specific contributions that reflect the purpose and direction of the ongoing conversation. Semantic theory describes these connections by expressing context-dependent meanings in terms of two constructs: anaphors, semantic variables that take on salient shared values; and presuppositions, conditions on the values of these variables that must be supported by salient facts from the context.

I have explored this theory to develop formal and computational accounts of the context-dependent interpretation of a range of forms in English, including disjunction, as realized by the English word or (Semantics and Linguistic Theory, 1992); signals of the evidential status of a conclusion, as realized in English by must (Computational Semantics Workshop, 1994); signals of temporal and hypothetical relationships within discourse, as realized by morphemes for tense and modality including would in English (Computing Meaning, 1999, with Daniel Hardt); and signals of other argumentative and domain connections between sentences in discourse, as realized by English markers such as for example, then and otherwise (as Bonnie Webber, Aravind Joshi and Alistair Knott and I describe in ACL 1999 and a paper to appear in Computational Linguistics).

My interest in such dependencies lies in how English speakers stitch together explanations of actions and plans. My 2000 Journal of Language and Computation paper provides an implemented case study of how such context-dependency can prove essential in providing agents with the information they need to act.

Current and Future Directions
Advances in computational models of language use set clear challenges and opportunities for the future of natural language research. By drawing on existing computational infrastructure, we can translate NL research results into engaging prototypes that motivate new, practical dialogue applications. My ongoing research continues my longstanding commitment to developing such prototypes (starting with my role in Cassell's Animated Conversation work reported to
SIGGRAPH and to the Cognitive Science Society in 1994, and continuing to a paper in the Computer Animation conference 2002 with colleagues Doug DeCarlo, Corey Revilla and Jennifer Venditti). At the same time, these testbeds make it possible to evaluate new NL modules and algorithms more thoroughly, in the context of working systems, and to further empirical investigations of language use by experimental methods and computational analysis. This methodological crossfertilization is in its infancy (Jennifer Venditti and I have initial results for the 2002 Conferences on Speech Prosody). But it suggests a future for the science of language in which a single program of investigation can not only improve the capabilities of conversational agents but can make a lasting contribution to the computational theory of human language use. This future depends on creating a broad literacy among junior researchers across disciplines. For my part, I am working to show the relevance of modern computational ideas in the study of language in writing for broad audiences: for philosophers (e.g., my chapter in What is Cognitive Science? Volume 2, 2003), for psychologists (e.g., my chapter in World-Situated Language Use 2002) and for linguists (e.g., my chapter in A Handbook for Language Engineers 2002).


Some cool new work