Synthetic speech has moved out of the imaginary realm of science fiction, where it provided the voices of robots from HAL to C-3PO, into our everyday world of appliances, telephony applications and personal communication devices. In the domain of assistive technology, individuals with severe communication disorders often rely on speech synthesis to speak on their behalf. Yet, anyone who has tried to listen to the PA system in a noisy train station or airport knows that speech synthesis fails in these everyday noise situations. These are precisely the types of communication scenarios in which users must rely most on the technology, thus highlighting the importance of developing a synthesizer capable of accommodating its speaking style to the user's situational noise context. In this exploratory project the PI will take first steps toward designing and developing a speech synthesizer that listens to the ambient noise level and modifies its prosody to compensate for background noise in human-like ways. The PI will begin by examining the modifications to speech in noise made by humans in order to learn how to implement these changes in machines. She posits that human modifications to speech will differ for semantically salient (i.e. content) words compared to non-salient (i.e. function) words; this is an important distinction that has not received much attention. If the hypothesis proves to be correct, the adaptive synthesizer will need to differentially modify its speaking style to account for the semantic role of the words spoken as well as the background noise level.
Broader Impacts: This research will impact all people who use spoken dialog interfaces (e.g., in telephony and virtual transaction applications), as well as individuals with disabilities who must rely on speech synthesis as their sole mode of communication. Most communicative interactions occur in lively spaces that have some degree of ambient noise. Current speech synthesis technology cannot cope with such conditions and thus it fails to serve as an enabling technology in commonplace situations such as classrooms, restaurants, hotel lobbies, cars, bus stations, and numerous other situations.
