"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."
The expression of affect in face-to-face situations requires the ability to generate a complex, coordinated, cross-modal affective signal, having gesture, facial expression, vocal prosody, and language content modalities. This ability is compromised in neurological disorders such as Parkinson?s disease and autism spectrum disorder (ASD). The PI's long term goal is to build computer-based interactive, agent based systems for remediation of poor affect communication and diagnosis of the underlying neurological disorders based on analysis of affective signals. A requirement for such systems is technology to detect atypical patterns in affective signals. The objective of this project is to develop that technology. Toward that end the PI will develop a play situation for eliciting affect, will collect audio-visual data from approximately 60 children between the ages of 4-7 years old, half of them with ASD and the other half constituting a control group of typically developing children. The PI will label the data on relevant affective dimensions, will develop algorithms for the analysis of affective incongruity, and will then test the algorithms against the labeled data in order to determine their ability to differentiate between ASD and typical development. While automatic methods for cross-modal recognition of discrete affect classes already have yielded promising results, automatic detection and quantification of atypical patterns in affective signals, and the ability to do so in semi-natural interactive situations, is unexplored territory. The PI expects this research will lead to new methods for affect recognition based on facial affective features (with special emphasis on facial frontalization algorithms and on modeling of facial expressive dynamics), vocal affective features, and lexical affective features, as well as to new methods for automated measurement of cross-modal affective incongruity.
Broader Impacts: The expression of affect in special populations is a largely neglected area in affective computing and robotics; yet, these populations may be among the most important beneficiaries of these technologies. Affective expression impairments afflict many individuals, including those with neuro-developmental disorders such as autism, and those with neuro-degenerative disorders such as Parkinson?s disease. Because these impairments concern a core aspect of human communication and, hence, may cause profound social isolation in these individuals, intervention is highly desirable. However, one-on-one intervention by therapists, if effective, would be available only to relatively few individuals, thereby making computer-based intervention critical for broader access to such treatment. Accurate processing of the affective signal will be of use as a research and diagnostic tool for a range of neurological disorders. The CSLU research team will continue its tradition of disseminating research findings and technology, including speech corpora and software, to the research community.
