Tactual perception of graphical images is a challenging problem. In this project, the PI and his team propose to develop new image processing and understanding techniques that specifically relate to low-bandwidth tactual perception. The PI and his team will design and build a prototype of an interactive Tactile Image Editor (TIE) that will enable sighted users to translate, as automatically as possible, graphical images into highly comprehensible Braille graphics (raised dots embossed on paper) that can be viewed tactually by blind persons. The focus is on graphical images (drawings, graphs, charts, and diagrams) that appear in textbooks, research papers, and Web pages that are used by scientists, engineers, and mathematicians (students and professionals). Criteria will be developed for what makes a Braille graphic as understandable as possible. That information will be provided to graphical image transcribers to enable them to make good decisions about the transcription of graphical images either manually or with the assistance of the TIE. To achieve his goals, the PI and his team will explore the use of image processing techniques to automate and expedite the conversion of graphical images that are in print or digital format into Braille graphics. They will explore application of low-level image processing techniques such as denoising, edge detection, smoothing, and resolution reduction, to determine heuristics for producing effective Braille graphics. They will validate developed heuristics with an experimental study, in which blind participants will evaluate the readability and comprehensibility of images produced by the derived heuristics. They will explore the application of segmentation techniques to enable them to locate text and other objects that need special attention within an image (for instance, text must be extracted and placed appropriately). They will develop classification and learning algorithms to enable them to determine the type of image being processed, because different kinds of images may require different processing steps. They will develop a software prototype, the Tactile Image Editor (TIE), which will enable them to easily apply the image processing heuristics, segmentation techniques, and classification and learning algorithms in future research. They will conduct an observational study of Braille transcribers, to better understand the processes they use and to inform the future design of the TIE's interface and functionality.
Broader Impacts: This research will lead to better and more timely access by blind scientists, engineers, and mathematicians (students and professionals) to graphical images that are becoming more and more prevalent in textbooks, research papers, and on the Web. The research will also lead to a better understanding of what makes Braille graphics most understandable; this information will be useful for graphical image transcribers, whether they use the TIE tool or not. Although the focus of the research is on scientific graphical material, the TIE is potentially useful in other areas that rely on graphical images to covey information. The students, both graduate and undergraduate, who work on the project will benefit from its cross-disciplinary nature.
