This revised proposal describes a project to systematically investigate the facial components, combinations of components, and interactions of components that constitute facial expressions (nonmanual markers) in the grammar of American Sign Language (ASL). Some of these components have already been shown to differ in significant ways from those used by the general hearing population. They may carry semantic, prosodic, pragmatic, and syntactic information that may not be provided by the manual signing itself. We will compile an inventory of facial articulations, construct a database of video images of these in isolation and in context, and use these data and innovative computational tools to construct a model of facial behavior in ASL. To successfully accomplish this, we propose an innovative integrated linguistic and computational approach to the study of nonmanuals. Our goal in this project is to construct an initial phonological model of ASL nonmanuals. We have targeted a relevant set of facial features and have identified 4 experiments to obtain appropriate information on each of them. A necessary step in preparation for these experiments is to develop computer vision and pattern recognition algorithms that automatically extract these facial features from a large quantity of videos. These algorithms will be capable of processing data more accurately and efficiently than can be done by hand. Finally, by comparing these results with those obtained from native ASL signers in a series of perceptual studies, we can determine what further modifications are still needed. The study of facial expressions in ASL has very practical applications to several areas affecting the lives of Deaf individuals. The absence of clear information on the facial components makes teaching them to individuals trying to learn ASL, such as parents, deaf children, future teachers and interpreters, a pedagogical nightmare. Another important practical application is the development of systems that automatically recognize ASL. Such a system is not feasible without the ability to handle ASL nonmanuals, which carry grammatical information. ? ?
Showing the most recent 10 out of 23 publications
