This revised proposal describes a project to systematically investigate the facial components, combinations of components, and interactions of components that constitute facial expressions (nonmanual markers) in the grammar of American Sign Language (ASL). Some of these components have already been shown to differ in significant ways from those used by the general hearing population. They may carry semantic, prosodic, pragmatic, and syntactic information that may not be provided by the manual signing itself. We will compile an inventory of facial articulations, construct a database of video images of these in isolation and in context, and use these data and innovative computational tools to construct a model of facial behavior in ASL. To successfully accomplish this, we propose an innovative integrated linguistic and computational approach to the study of nonmanuals. Our goal in this project is to construct an initial phonological model of ASL nonmanuals. We have targeted a relevant set of facial features and have identified 4 experiments to obtain appropriate information on each of them. A necessary step in preparation for these experiments is to develop computer vision and pattern recognition algorithms that automatically extract these facial features from a large quantity of videos. These algorithms will be capable of processing data more accurately and efficiently than can be done by hand. Finally, by comparing these results with those obtained from native ASL signers in a series of perceptual studies, we can determine what further modifications are still needed. The study of facial expressions in ASL has very practical applications to several areas affecting the lives of Deaf individuals. The absence of clear information on the facial components makes teaching them to individuals trying to learn ASL, such as parents, deaf children, future teachers and interpreters, a pedagogical nightmare. Another important practical application is the development of systems that automatically recognize ASL. Such a system is not feasible without the ability to handle ASL nonmanuals, which carry grammatical information. ? ?

Agency
National Institute of Health (NIH)
Institute
National Institute on Deafness and Other Communication Disorders (NIDCD)
Type
Research Project (R01)
Project #
5R01DC005241-03
Application #
6985348
Study Section
Biobehavioral and Behavioral Processes 3 (BBBP)
Program Officer
Cooper, Judith
Project Start
2004-01-01
Project End
2008-11-30
Budget Start
2005-12-01
Budget End
2006-11-30
Support Year
3
Fiscal Year
2006
Total Cost
$475,410
Indirect Cost
Name
Purdue University
Department
Other Health Professions
Type
Schools of Arts and Sciences
DUNS #
072051394
City
West Lafayette
State
IN
Country
United States
Zip Code
47907
Malaia, Evie; Wilbur, Ronnie B; Weber-Fox, Christine (2012) Effects of verbal event structure on online thematic role assignment. J Psycholinguist Res 41:323-45
Malaia, Evie; Ranaweera, Ruwan; Wilbur, Ronnie B et al. (2012) Event segmentation in a visual language: neural bases of processing American Sign Language predicates. Neuroimage 59:4094-101
Malaia, Evie; Wilbur, Ronnie B (2012) Kinematic signatures of telic and atelic events in ASL predicates. Lang Speech 55:407-21
You, Di; Hamsici, Onur C; Martinez, Aleix M (2011) Kernel optimization in discriminant analysis. IEEE Trans Pattern Anal Mach Intell 33:631-8
Gotardo, Paulo F U; Martinez, Aleix M (2011) Computing Smooth Time Trajectories for Camera and Deformable Shape in Structure from Motion with Occlusion. IEEE Trans Pattern Anal Mach Intell 33:2051-65
Fortuna, Jeff; Martinez, Aleix M (2010) Rigid Structure from Motion from a Blind Source Separation Perspective. Int J Comput Vis 88:404-424
Ding, Liya; Martinez, Aleix M (2010) Features versus context: An approach for precise and detailed detection and delineation of faces and facial features. IEEE Trans Pattern Anal Mach Intell 32:2022-38
Malaia, Evie; Wilbur, Ronnie B (2010) Early acquisition of sign language What neuroimaging data tell us. Sign Lang Linguist 13:183-199
Malaia, Evie; Wilbur, Ronnie (2010) Sign Languages: Contribution to Neurolinguistics from Cross-Modal Research. Lingua 120:2704-2706
Wilbur, Ronnie B (2009) Effects of varying rate of signing on ASL manual signs and nonmanual markers. Lang Speech 52:245-85

Showing the most recent 10 out of 23 publications