Spoken word recognition is defined as the process by which acoustic patterns are matched to meanings in the "mental lexicon" -- the memory repository of the approximately 85000 words known by an adult language speaker. Previous research has demonstrated that the number of lexical entries acoustically similar to a given word influences the ease with which that word is recognized. However, speech recognition is not solely an acoustic phenomenon; watching someone speak also provides additional information about the content of an utterance. Although there is presently a solid understanding of the role of acoustic similarity in spoken word recognition, less is known about visual similarity, and very little is known about the interaction of acoustic and visual similarity when both sources of information are available. This may be because visual similarity is particularly difficult to measure. One problem is that speech units that are acoustically different can be visually identical. Words that are visually identical are said to comprise a Lexical Equivalence Class (LEC). Previous research has shown that the number of words residing in an LEC affects the ease with which those words are lipread. Similarly, the size of an LEC affects the extent to which visual information enhances the recognition of auditory speech. However, the makeup of an LEC is to some extent dependent on the speaker. The proposed research will accomplish two goals: first, a publicly accessible computational tool will be built to facilitate computational and experimental investigations of visual lexical similarity. Second, several behavioral experiments will further elucidate the role that visual similarity plays in spoken word recognition. This project will advance our understanding of the basic mechanisms involved in spoken word recognition. Such knowledge will be useful for clinicians working with deaf or hearing-impaired populations, and engineers working on problems in automatic speech recognition.
