The goal of this project is to develop, implement, and evaluate an assistive hearing system that isolates sound sources that are located in the direction of the user?s eye gaze. The system is meant to be used in conjunction with a binaural hearing aid to improve speech perception for the hearing impaired population. Poor speech perception is a frequent complaint of those suffering from hearing loss and is associated with an increased risk of dementia, depression, and feelings of isolation in older adults. Perceiving a speech stream in the presence of competing speech is often especially difficult for hearing impaired listeners, and traditional hearing aids do not do much to solve the problem. Previous attempts to address this issue have often relied upon directional microphones that are oriented to pick up sounds coming from directly in front of the user. However, the focus of attention often deviates away from directly in front of the listener, especially in auditory environments with many sound sources. Instead of a system focused on a fixed location, we propose a highly innovative directional listening system that uses a microphone array to isolate sounds coming from the focus of the user?s attention. The major hypotheses of this work are that eye gaze direction is a strong indicator of the focus of auditory attention, and that a gaze-controlled directional listening system represents an effortless and effective way to isolate the speech signals that the listener cares about. The project?s first specific aim is to refine an existing and highly promising directional listening algorithm to maximize its ability to isolate a target speech signal. Human subjects testing will be used to evaluate many different revisions of the algorithm according to the speech fidelity provided and listening effort required. The highest-performing version of the algorithm will be implemented as part of the gaze-controlled system. Concurrent with this effort, the project?s second specific aim is to determine how gaze information can be best used to control the directional listening system. More generally, the goal is to determine how the focus of visual attention corresponds to the focus of auditory attention in normal and hearing impaired subjects.
This aim i ncludes a series of basic research studies in which eye movements are monitored while subjects view complex audiovisual speech stimuli. These data will suggest the best way to translate eye tracking data into an estimate of the focus of auditory attention, while also providing results relevant to the multisensory attention literature. Finally, the major goal of third specific aim is to conduct a comprehensive test of the gaze-controlled directional listening system with hearing impaired subjects. Subjects will complete listening tasks in auditory and visual environments of varying complexity. Longer-term testing will also be completed to evaluate system performance with experienced users. Ultimately, the project is expected to result in a highly effective and innovative approach to allow the user to hear what they see.
? Attention-guided speech enhancement for hearing impaired listeners This project is focused on the development and evaluation of a gaze-controlled speech enhancement system. The system uses gaze orientation to control the directionality of a microphone array, thereby isolating sound sources that are located in the area of visual attention. The goal is to enhance speech reception and lower listening effort for hearing impaired listeners.
