For individuals who suffer form extreme paralysis, the ability to communicate is often limited to yes/no responses using small head, hand, or eye movements. Providing these people with the ability to participate in the Information Society (for example, by browsing the Web, posting messages, or emailing friends), would greatly enhance their emotional well being by alleviating the frustration caused by having an active mind trapped in a paralyzed body. Previously, the PI has developed camera-based interfaces that have afforded partial attainment of such goals. In the current project, the PI will build upon her prior achievements and push forward on several fronts concurrently. She will develop a mouse-replacement interface that computes pointer coordinates from the user's head movements, which are detected by a multi-camera computer vision system. She will explore context-aware approaches for gesture detection that can distinguish a user's communicative motions from involuntary movements or social interactions. She will design and implement a novel tool that allows users to develop their own semaphore-based communication interfaces. She will build an application mediator that serves as an intercessor between several separate software components. And she will develop innovative assistive software for a variety of common applications including text entry, web browsing, and animation (in the latter case, which enables users capable only of controlling a mouse cursor with limited precision to create and interact with 3D objects).
Broader Impacts: Project outcomes will have a direct and positive impact on the quality of life of adults and children with severe disabilities, as well as their friends, families and caregivers. The software to be developed will be disseminated at special care facilities (including a hospital, schools for children with severe physical disabilities, and a long-term care facility for people with Multiple Sclerosis and ALS), and will also be available on the internet via free download. This work will also advance the state of the art in computer vision, through the development of technology that employs 3D models for real-time tracking of facial features with multiple pan/tilt/zoom cameras.
