PI: Borkholder, D. A., Amuso, V. J., Eddins, D. A. Proposal Number: 0967732
The research objective of this proposal is to test the hypothesis that an assistive listening system with dynamic, user-controlled directional characteristics will improve communication for the hearing impaired significantly relative to traditional fixed-directional systems. The proposed study will remove barriers to the delivery of acoustic information to people who are deaf or hard of hearing, enhancing their participation in education, work, and social settings, and improving their quality of life. Interfering speech, reverberation, and temporally-fluctuating background noise have a profound impact on speech perception for the hearing impaired. Directional microphone arrays have been shown to improve key metrics of speech intelligibility, especially for the hearing impaired, but minimal user guidance capabilities limit achievable benefits in multi-talker situations. It is widely accepted that current personal assistive listening systems are inadequate in small-group, multi-talker settings. The proposed research will explore user controlled directivity coupled with speaker identification algorithms and tracking techniques from the radar and infrared tracking communities to enable an advanced user-guided assistive listening system. Acoustic beamforming and parallel processing offer opportunities for users to control a focal acoustic zone while attenuating noise sources outside of this zone. A hybrid pattern recognition scheme incorporating several techniques for parameter extraction and speaker identification will allow robust dynamic identification of speakers in multi-talker situations. Target tracking algorithm will dynamically steer the acoustic focal zone as the talker moves through the meeting area, relieving the listener of the burden of manual tracking. A user interface will provide spatial information for each identified talker, allowing the listener to independently select the desired talker, or manually steer the acoustic beam to a desired location. The study will involve exploration of the most effective parameters for user-control, algorithms for phased-array beamforming, speaker identification, and target tracking for this acoustic application, and quantitative hypothesis testing with hearing impaired listeners in controlled environments. This will be a collaborative effort between the Rochester Institute of Technology College of Engineering, College of Science, the National Technical Institute for the Deaf, the International Center for Hearing and Speech Research, and the University of Rochester - Medical Center Department of Otolaryngology. The research will involve engineering graduate students, AALANA and women undergraduate students, and hearing impaired individuals from NTID, allowing a unique opportunity to foster an appreciation of technology to aid persons with disabilities. Intellectual Merit The novelty of the proposed research lies in the integration of technologies and approaches from different fields to address a critical need of the hearing impaired, and testing the hypothesis that a shift from technology-control to user-control of parameters enhances communication By incorporating speaker identification with target tracking and user-guidance of an acoustic focal zone, the technology is expected to outperform current state-of-the-art assistive listening systems in the small group, multi-talker circumstance in terms of speech intelligibility metrics. The investigators bring a distinctive combination of expertise in the areas of biomedical systems, radar, psychoacoustics and assistive listening devices, providing a unique opportunity for interdisciplinary innovation in technology to aid the hearing impaired. Broader Impacts The proposed research will improve the ability of hard-of-hearing individuals to participate in education, work and social settings by enabling a presently unavailable level of user control critical for the multitalker situation. It will enhance education opportunities for the hearing impaired and may allow more students to enter the STEM disciplines. Demonstration that user control of assistive listening device parameters enhances communication could have a profound impact on the philosophy and approaches for other assistive listening devices such as hearing aids and cochlear implants. The proposed activities integrate research with education for graduate and undergraduate engineering students, under-represented minorities, and hearing impaired individuals. The results will be broadly disseminated to the engineering and disability communities through conferences and journals, and will leverage the expertise of Center on Access Technology for integration into the educational mission of this and other universities.
This research aimed to remove barriers to the delivery of acoustic information to people who are deaf or hard of hearing, enhancing their participation in education, work, and social settings, and improving their quality of life. An assistive listening system with dynamic, user-controlled directional characteristics was developed and tested. A custom microphone array with integrated signal processing achieved directional characteristics through classical acoustic beamforming providing a focal acoustic zone and attenuating noise and competing speech outside of this zone. A touchscreen software interface enabled user control of the listening zone(s) enabling a new level of control in multi-talker situations. Through this research a new metric, the focal index (FI), was developed to quantitatively gauge the ability of a beampattern to perform spatial filtering, while the speech-intelligibility-weighted FI (SII-FI) was created to evaluate the performance of the beamforming system as a whole across the frequency range of operation. The FI was shown to be the most robust metric for quantifying the performance of an individual beampattern, in comparison with existing metrics such as beamwidth (BW), peak side-lobe level (PSL), integrated side-lobe level (ISL), and planar directivity index (PDI). Using the SII-FI to drive array parameter selection, a 16 by 16 gridded microphone array with Taylor weighting was selected for the user guided assistive listening system (UGALS) prototype. Based on the results of human subject testing, the SII-FI metric was determined to be linearly related to intelligibility scores derived from the simulated multi-talker environment. This relationship is very valuable as it allows microphone array designers to gain intuition regarding anticipated system performance strictly through simulation while avoiding rounds of human subject testing. The FI was instrumental in gauging the ability of the array to perform spatial filtering by providing a comprehensive single value of merit. The UGALS system is now located at the University of South Florida Global Center for Hearing and Speech Research. It will be used in future studies more fully exploring the improvements in communication for the hearing impaired provided by an assistive listening system with dynamic, user-controlled directional characteristics. Intellectual Merit The novelty of this research lies in the integration of technologies and approaches from different fields to address a critical need of the hearing impaired. The multidisciplinary investigative team leveraged their expertise in biomedical systems, radar, psychoacoustics, and assistive listening devices to create this innovative technology to aid the hearing impaired. Broader Impacts The developed system has the potential to improve the ability of hard-of-hearing individuals to participate in education, work and social settings by enabling a previously unavailable level of user control critical for the multi-talker situation. The technology may enhance education opportunities for the hearing impaired and may allow more students to enter the STEM disciplines. As system use is explored in more environments outside of the laboratory, it ultimately may foster shifts in philosophy and approaches for other assistive listening devices such as hearing aids and cochlear implants where user control of listening direction is currently not available. This research also engaged deaf faculty and undergraduate under-represented and deaf students in the system design.
