Our goal is to construct computer vision systems to enable the blind and severely visually impaired to detect and read informational text in city scenes. The informational text can be street signs, bus numbers hospital signs, supermarket signs, and names of products (eg. Kellogg's cornflakes). We will construct portable prototype computer vision systems implemented by digital cameras attached to personal, or hand held, computers. The camera need only be pointed in the general direction of the text (so the text is only one percent of the image). A speech synthesizer will read the text to the user. Blind and visually impaired users will test the device in the field (under supervision) and give feedback to improve the algorithms. We argue that this work will make a significant contribution to improving human health (rehabilitation). Computer vision is a rapidly maturing technology with immense potential to help the blind and visually impaired. Reports suggest that detecting and reading informational text is one of the main unsatisfied desires of these groups. Written signs and information in the environment are used for navigation, shopping, operating equipment, identifying buses, and many other purposes (to which a blind person does not otherwise have independent access). The blind and severely visually impaired make up a large fraction of the US population (3 million). Moreover, this proportion is expected to increase by a factor of two in the next ten years due to increased life expectancy. Our proposal is design-driven. It uses a new class of computer vision algorithms known as Data Driven Monte Carlo (DDMCMC). The algorithms are used to: (i) search for text, and (ii) to read it. Recent developments in digital cameras and portable/handheld computers make it practical to implement these algorithms in portable prototype systems. The three scientists in this proposal have the necessary expertise to accomplish it. Dr.'s Yuille and Zhu have backgrounds in computer vision and Dr. Brabyn has experience in developing and testing engineering systems to help the blind and visually impaired. Our proposal falls within the scope of the Bioengineering initiative because we are applying techniques from the mathematical/engineering sciences to develop informatic approaches for patient rehabilitation. More specifically, our work will facilitate the development of portable devices to help the blind and visually impaired.
| Chen, Hong; Zhu, Song-Chun (2006) A generative sketch model for human hair analysis and synthesis. IEEE Trans Pattern Anal Mach Intell 28:1025-40 |
| Kersten, Daniel; Mamassian, Pascal; Yuille, Alan (2004) Object perception as Bayesian inference. Annu Rev Psychol 55:271-304 |
