A Cell Phone-Based Street Intersection Analyzer for Visually Impaired Pedestrians Abstract Project Summary Urban intersections are among the most dangerous parts of a blind person's travel. They are becoming increasingly complex, making safe crossing using conventional blind orientation and mobility techniques ever more difficult. To alleviate this problem, we propose to develop and evaluate a cell phone-based system to analyze images of street intersections, taken by a blind or visually impaired person using a standard cell phone, to provide real-time feedback. Building on our past work on a prototype "Crosswatch" system that uses computer vision algorithms to find crosswalks and Walk lights, we will greatly enhance the functionality of the system with information about the intersection layout and the identity of its connecting streets, the presence of stop signs, one-way signs and other controls indicating right-of-way, and timing information integrated from Walk/Don't Walk lights, countdown timers and other traffic lights. The system will convey intersection information, and will actively guide the user to align himself/herself with crosswalks, using a combination of synthesized speech and audio tones. We will conduct human factors studies to optimize the system functionality and the configuration of the user interface, as well as develop interactive training applications to equip users with the skills to better use the system. These training applications, implemented as additional cell phone software to complement the intersection system, will train users to hold the camera horizontal and forward and to minimize veer when traversing a crosswalk. The intersection analysis and training software will be made freely available for download onto popular cell phones (such as iPhone, Android or Symbian models). The cell phone will not need any hardware modifications or add-ons to run this software. Ultimately a user will be able to download an entire suite of such algorithms for free onto the cell phone he or she is already likely to be carrying, without having to carry a separate piece of equipment for each function.

Public Health Relevance

The ability to walk safely and confidently along sidewalks and traverse crosswalks is taken for granted every day by the sighted, but approximately 10 million Americans with significant vision impairments and a million who are legally blind face severe difficulties in this task. The proposed research would result in a highly accessible system (with zero or minimal cost to users) to augment existing wayfinding techniques, which could dramatically improve independent travel for blind and visually impaired persons.

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Special Emphasis Panel (ZRG1-ETTN-E (92))
Program Officer
Wiggs, Cheri
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Smith-Kettlewell Eye Research Institute
San Francisco
United States
Zip Code
Fusco, Giovanni; Shen, Huiying; Coughlan, James M (2014) Self-Localization at Street Intersections. Proc Conf Comput Robot Vis :40-47
Coughlan, James M; Shen, Huiying (2013) Crosswatch: a System for Providing Guidance to Visually Impaired Travelers at Traffic Intersections. J Assist Technol 7:
Ivanchenko, Volodymyr; Coughlan, James; Shen, Huiying (2010) Real-Time Walk Light Detection with a Mobile Phone. Lect Notes Comput Sci 6180:229-234
Shen, Huiying; Coughlan, James; Ivanchenko, Volodymyr (2009) Figure-Ground Segmentation Using Factor Graphs. Image Vis Comput 27:854-863
Ivanchenko, V; Coughlan, J; Shen, H (2009) Staying in the Crosswalk: A System for Guiding Visually Impaired Pedestrians at Traffic Intersections. Assist technol Res Ser 25:69-73
Ivanchenko, Volodymyr; Coughlan, James; Shen, Huiying (2008) Crosswatch: a Camera Phone System for Orienting Visually Impaired Pedestrians at Traffic Intersections. Lect Notes Comput Sci 5105:1122-1128
Ivanchenko, Volodymyr; Coughlan, James; Shen, Huiying (2008) Detecting and Locating Crosswalks using a Camera Phone. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2008:4563143