Penn Arts & Sciences Logo


Friday, February 19, 2016 - 12:00pm

IRCS Conference Room

Emily Mower Provost
CHAI Laboratory -- Computational Human-Centered Analysis and Integration
University of Michigan

Human-Centered Computing: Using Speech to Understand Behavior

Emotion has intrigued researchers for generations. This fascination has permeated the engineering community, motivating the development of affective computational models for classification. However, human emotion remains notoriously difficult to interpret in part due to the presence of complex emotions, emotions that contain shades of multiple affective classes. Proper representations of emotion would ameliorate this problem by introducing multidimensional characterizations of the data that permit the quantification and description of the varied affective components of each utterance. In this talk I will discuss methods to characterize emotion, focusing on quantifying the presence of multiple shades of affect and avoiding the need for hard-labeled assignments. This set of techniques can be used to determine a most likely assignment for an utterance, to map out the evolution of the emotional tenor of an interaction, or to interpret utterances that have multiple affective components. I will demonstrate how these representation techniques can be used as a component of classification and how they provide insight into the temporal flow of emotion in speech.

I will also highlight our ongoing speech-based assistive technology research, estimating speech quality for individuals with aphasia. I will describe our interactive tablet-based software, which is based on picture description tasks, and illustrate how we have used this platform to collect a new dataset of aphasic and healthy speech. We have demonstrated that we can use these data to automatically estimate speech quality at levels comparable to an average human evaluator. Finally, I will touch on our work classifying mood for individuals with bipolar disorder using naturally collected cell phone data.