Keyboards on mobile devices display characters visually, and in so doing they present challenges for people who are blind as well as for all users in situations where it is inconvenient or unsafe to hold a phone in order to communicate. Existing approaches to mobile accessibility for eyes-free text entry do not fully solve this problem, because they remain screen-centric; manipulating text generally requires interacting with visual keypads that at most read aloud keys upon touching. A major unsolved challenge is how to manipulate text aurally and silently, in a way that unbinds users from a visual display. The goal of this project is to expand our understanding of text interaction from a screen-centric paradigm to screenless, aural environments. To this end, the work will establish and evaluate principles for aural text manipulation that do not rely on a reference screen but rather operate entirely over the auditory channel. This research is significant because it is expected to contribute novel strategies to augment the ability of over seven million blind people in the United States to perform text-entry by circumventing direct interaction with screen-based devices. By breaking free from the constraints of mobile visual displays, project outcomes will have broader applicability to support aural text manipulation for sighted users in situations where it is inconvenient or unsafe to stay glued to the screen while typing, or where text manipulation needs to be discreet, silent, and concealed from view. The project will directly engage people who are blind and visually impaired from three partner organizations in Indiana as well as students at Indiana University from underrepresented groups to co-create and evaluate new options to interact with text while bypassing the screen.

The project will pursue two technical thrusts. The first thrust will identify strategies for the auditory arrangement of the character set that operate over time rather than within the visual-spatial constraints of the screen. The conceptual advances will be designed to work with current and future wearable input devices (hand/finger gestures supported by armband or smart rings) and will inform the foundation for a new class of auditory keyboards untethered to a visual display. The second thrust will focus on conducting iterative and comparative evaluation studies with sighted and blind participants on system prototypes of auditory keyboards, in order to examine how people can aurally manipulate text in screenless contexts. By exploring and validating novel principles to re-imagine the concept of keyboard in the aural modality, the project will contribute to shifting the way people think about accessible typing in eyes-free scenarios and will play a fundamental role in the understanding of how people respond to screenless environments for text manipulation.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Agency
National Science Foundation (NSF)
Institute
Division of Information and Intelligent Systems (IIS)
Type
Standard Grant (Standard)
Application #
1909845
Program Officer
Ephraim Glinert
Project Start
Project End
Budget Start
2019-10-01
Budget End
2022-09-30
Support Year
Fiscal Year
2019
Total Cost
$499,876
Indirect Cost
Name
Indiana University
Department
Type
DUNS #
City
Bloomington
State
IN
Country
United States
Zip Code
47401