Access to mathematics education is critical to facilitating careers in the sciences, engineering, technology, and in many professions, as well as to providing the mathematical literacy needed in everyday life. The PI's goal in this project is to increase real-time access in the classroom to mathematical material for low vision students (who are visually impaired but not blind), which is currently mostly lacking as these students must primarily rely on assistive technology and human note takers to provide access to course materials after the class meeting. The PI's approach is to bring together expertise in computer science, mathematics, and accessibility to provide a useful and usable solution to this problem for the iPad. Called AccessMath, the prototype application will provide zoom-able camera views as well as a view of notes written at the whiteboard (captured by a Mimio), which may be altered through magnification and contrast adjustment features. The system will support consulting and searching course content during lecture, including textbooks, handouts, notes written at the whiteboard, and audio in video (e.g., to find the word "eigenvalue" in recorded lectures for the course). A novel note-taking facility will be employed, where notes are represented on note "cards" that can be attached to a specific location in the current view (e.g., at a spot in the whiteboard view), then manually moved, resized, rotated, stacked, and "flicked" out of view, providing a natural means for interacting informally with notes created during lecture. Notes are created as blank cards, or as cards containing views of whiteboard data, video excerpts, or existing notes. Notes are also used in constructing search queries. Linear algebra (in particular matrices) is the pilot scope for the project, due to the many challenges in conveying a matrix, individual items within the matrix, following derivations, and the scale of the matrix needed when completing non-trivial problems. AccessMath users will have a single iPad which connects to a server providing video feeds and recognition and retrieval services.
Broader Impacts: AccessMath will assist low-vision students with mathematical instruction in and out of class, at the high school and university levels, by providing immediate access to lecture materials, including searches for mathematical content in whiteboard notes, handouts, textbooks, and lecture videos (through word spotting in audio). The system will afford student ownership in note taking and customizing access, which will be significantly better than current solutions, and will reduce or eliminate the extent to which visually impaired students fall behind their sighted peers in the classroom setting. The project will also advance the state-of-the-art in recognition and retrieval for mathematical notation, and will create a novel note-taking model using the metaphor of photos on a tabletop, which should be applicable to domains other than mathematics.
