The goal of this project is to develop a computer vision system based on standard camera cell phones to give blind and visually impaired persons the ability to read appliance displays and similar forms of non-document visual information. This ability is increasingly necessary to use everyday appliances such as microwave ovens and DVD players, and to perform many daily activities such as counting paper money. No access to this information is currently afforded by conventional text reading systems such as optical character recognition (OCR), which is intended for reading printed documents. Our proposed software runs on a standard, off-the- shelf camera phone and uses computer vision algorithms to analyze images taken by the user, to detect and read the text within each image, and to then read it aloud using synthesized speech. Preliminary feasibility studies indicate that current cellular phones easily exceed the minimum processing power required for these tasks. Initially, the software will read out three categories of symbols: LED/LCD appliance displays, product or user-defined barcodes, and denominations of paper money. Ultimately these functions will be integrated with other capabilities being developed under separate funding, such as reading a broad range of printed text (including signs), recognizing objects, and analyzing photographs and graphics, etc., all available as free or low-cost software downloads for any cell phone user. Our specific goals are to (1) gather a database of real images taken by blind and visually impaired persons of a variety of LED/LCD appliance displays, barcodes and US paper currency; (2) develop algorithms to process the images and extract the desired information; (3) implement the algorithms on a camera phone; and (4) conduct user testing to establish design parameters and optimize the human interface.
For blind and visually impaired persons, one of the most serious barriers to employment, economic self sufficiency and independence is insufficient access to the ever-increasing variety of devices and appliances in the home, workplace, school or university that incorporate visual LED/LCD displays, and to other types of text and symbolic information hitherto unaddressed by rehabilitation technology. The proposed research would result in an assistive technology system (with zero or minimal cost to users) to provide increased access to such display and non- document text information for the approximately 10 million Americans with significant vision impairments or blindness. ? ? ?
| Coughlan, James M; Miele, Joshua (2017) Evaluating Author and User Experience for an Audio-Haptic System for Annotation of Physical Models. ASSETS 2017:369-370 |
| Fusco, Giovanni; Tekin, Ender; Giudice, Nicholas A et al. (2015) Appliance Displays: Accessibility Challenges and Proposed Solutions. ASSETS 2015:405-406 |
| Fusco, Giovanni; Tekin, Ender; Ladner, Richard E et al. (2014) Using Computer Vision to Access Appliance Displays. ASSETS 2014:281-282 |
| Tekin, Ender; Vásquez, David; Coughlan, James M (2013) S-K Smartphone Barcode Reader for the Blind. J Technol Pers Disabil 28:230-239 |
| Manduchi, Roberto; Coughlan, James (2012) (Computer) Vision without Sight. Commun ACM 55:96-104 |
| Tekin, Ender; Coughlan, James M; Shen, Huiying (2011) Real-Time Detection and Reading of LED/LCD Displays for Visually Impaired Persons. Proc IEEE Workshop Appl Comput Vis :491-496 |
| Tekin, Ender; Coughlan, James M (2010) A Mobile Phone Application Enabling Visually Impaired Users to Find and Read Product Barcodes. Comput Help People Spec Needs 6180:290-295 |
| Tekin, Ender; Coughlan, James M (2010) A Mobile Phone Application Enabling Visually Impaired Users to Find and Read Product Barcodes. Lect Notes Comput Sci 6180:290-295 |
| Tekin, Ender; Coughlan, James (2009) A Bayesian Algorithm for Reading 1D Barcodes. Proc Can Conf Comput Robot Vis 2009:61-67 |
| Tekin, Ender; Coughlan, James M (2009) An Algorithm Enabling Blind Users to Find and Read Barcodes. Proc IEEE Workshop Appl Comput Vis 2009:1-8 |
