Abstract

There are more than 10 million blind and visually impaired people living in America today. Recent technology developments in computer vision, digital cameras, and portable computers make it possible to assist these individuals by developing camera-based products that combine computer vision technology with other existing products. Although a number of reading assistants have been designed specifically for people who are blind or visually impaired, reading text from complex backgrounds or non-flat surfaces is very challenging and has not yet been successfully addressed. Many everyday tasks involve these challenging conditions, such as reading instructions on vending machines, titles of books aligned on a shelf, instructions on medicine bottles or labels on soup cans. This proposal focuses on the development of new computer vision algorithms to recognize text from complex backgrounds: 1) from backgrounds with multiple different colors (e.g .. the titles of books lined up on a shelf) and 2) from non-flat surfaces (e.g .. labels on medicine bottles or soup cans). The newly developed computer vision techniques will be integrated with off-the-shelf optical character recognition (OCR) and speech-synthesis software products. Visual information will be captured via a head-mounted camera (on sunglasses or hat) and analyzed by a portable computer (PDA or cell phone), while the speech display will be outputted via mini speakers, earphones, or Bluetooth device. A practical reading system prototype will be produced to read text from complex backgrounds and non-flat surfaces. The system will be cost-effective since it requires only a head mounted camera (

Public Health Relevance

The goal of the proposed research is to develop new computer vision algorithms for camera-based text recognition from complex backgrounds and non-flat surfaces, as well as produce a practical reading system prototype in combination with off-the-shelf optical character recognition (OCR) and speech-synthesis software products, to help blind or visually impaired people read instructions on vending machines, titles of books aligned on a shelf, labels on medicine bottles or soup cans, etc. Visual information will be captured via a head-mounted camera (on sunglasses or hat) and analyzed in realtime through a portable computer, such as a mini laptop or a personal digital assistant (PDA). The speech display will be outputted via mini speakers, earphones, or Bluetooth device.

  Funding Agency

Agency
National Institute of Health (NIH)
Institute
National Eye Institute (NEI)
Type
Exploratory/Developmental Grants (R21)
Project #
1R21EY020990-01
Application #
7977496
Study Section
Special Emphasis Panel (ZRG1-ETTN-E (92))
Program Officer
Wiggs, Cheri
Project Start
2010-09-01
Project End
2012-08-31
Budget Start
2010-09-01
Budget End
2012-08-31
Support Year
1
Fiscal Year
2010
Total Cost
$190,000
Indirect Cost

  Institution

Name
City College of New York
Department
Engineering (All Types)
Type
Schools of Engineering
DUNS #
603503991
City
New York
State
NY
Country
United States
Zip Code
10031

  Publications

Yang, Xiaodong; Tian, Yingli (2013) Texture Representations Using Subspace Embeddings. Pattern Recognit Lett 34:1130-1137
Hasanuzzaman, Faiz M; Yang, Xiaodong; Tian, Yingli et al. (2013) Monitoring Activity of Taking Medicine by Incorporating RFID and Video Analysis. Netw Model Anal Health Inform Bioinform 2:61-70
Yi, Chucai; Flores, Roberto W; Chincha, Ricardo et al. (2013) Finding Objects for Assisting Blind People. Netw Model Anal Health Inform Bioinform 2:71-79
Wang, Shuihua; Yang, Xiaodong; Tian, Yingli (2013) Detecting Signage and Doors for Blind Navigation and Wayfinding. Netw Model Anal Health Inform Bioinform 2:81-93
Tian, Yingli; Yang, Xiaodong; Yi, Chucai et al. (2013) Toward a Computer Vision-based Wayfinding Aid for Blind Persons to Access Unfamiliar Indoor Environments. Mach Vis Appl 24:521-535
Yi, Chucai; Tian, Yingli (2013) Text Extraction from Scene Images by Character Appearance and Structure Modeling. Comput Vis Image Underst 117:182-194
Hasanuzzaman, Faiz M; Yang, Xiaodong; Tian, Yingli (2012) Robust and Effective Component-based Banknote Recognition for the Blind. IEEE Trans Syst Man Cybern C Appl Rev 42:1021-1030
Yi, Chucai; Tian, Yingli (2012) Localizing text in scene images by boundary clustering, stroke segmentation, and string fragment classification. IEEE Trans Image Process 21:4256-68
Yi, Chucai; Tian, YingLi (2011) Text string detection from natural scenes by structure-based partition and grouping. IEEE Trans Image Process 20:2594-605
Hasanuzzaman, Faiz M; Yang, Xiaodong; Tian, YingLi (2011) Robust and Effective Component-based Banknote Recognition by SURF Features. WOCC 2011:1-6

Showing the most recent 10 out of 13 publications