Deb Roy
Situated Language Use: Robots, Children, and Games
Our lab aims to create machines that use language in human-like ways, and to better understand and model human-human communication using new methods of observation and analysis. In the first part of this talk, I will develop a model of how referential semantics is grounded in physical interaction based on experiments with conversational robots [1,2]. Second, I will discuss the capture and analysis of communicative “interaction traces” of adults engaged in two-player computer games [3,4], and of child development based on rich longitudinal /in vivo/ observation [5].
References
1. Deb Roy. (in press). A Mechanistic Model of Three Facets of Meaning. In /Symbols, Embodiment, and Meaning/, de Vega, Glenberg, and Graesser, eds.
2. Deb Roy. (2005). Semiotic Schemas: A Framework for Grounding Language in Action and Perception. /Artificial Intelligence/, 167(1-2):170-205.
3. Peter Gorniak and Deb Roy. (2007). Situated Language Understanding as Filtering Perceived Affordances. /Cognitive Science/, 31(2), 197-231.
4. Jeff Orkin and Deb Roy. (in press). The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online. /Journal of Game Development./
5. Deb Roy, Rupal Patel, Philip DeCamp, Rony Kubat, Michael Fleischman, Brandon Roy, Nikolaos Mavridis, Stefanie Tellex, Alexia Salata, Jethran Guiness, Michael Levit, Peter Gorniak. (2006). The Human Speechome Project. /Twenty-eighth Annual Meeting of the Cognitive Science Society/ /Papers are available online at http://www.media.mit.edu/cogmac/publications.html./