To overcome accessibility problems, people with disabilities have traditionally relied upon the support of others in their community. For instance, a volunteer may offer a few minutes of her time to read a blind person's mail aloud, or a fellow traveler may answer a quick question at the bus stop (e.g., "Is that the 45 coming?"). Professionals such as sign language interpreters and narrators of audio descriptions convert sensory information into alternative forms that enable a deaf student to participate in a conventional lecture and a blind person to enjoy a movie. Internet connectivity has dramatically expanded the pool of potential human supporters, but finding reliable assistance on demand remains difficult. The PI's goal in this project is to enable dynamic and diverse groups of people reachable via the Web ("the crowd") to interactively support people with disabilities. The crowd is whoever happens to be available, from paid workers recruited on burgeoning micro-task marketplaces, to friends and family recruited via existing social networks, to volunteers willing to give a few minutes of their time. While someone is always available, the crowd is dynamic and individual workers can be unreliable. Thus, developing interactive systems to support people with disabilities presents numerous challenges, including how to enable crowd support that is real-time and high-quality, how to design interfaces that provide effective feedback even when an individual's directions may not be followed, and how to support collaboration among individual crowd workers without subverting methods of ensuring reliability. To these ends, the PI will explore closed-loop crowd support with a number of applications. VizWiz Stream will enable blind users to engage in an interactive conversation with the crowd about their visual environment. AudioWiz Stream will provide nearly real-time transcription of aural speech. And Legion and Legion VM will enable the crowd to collectively assume control of the keyboard and mouse to complete a user-specified task on existing desktop interfaces (whereas research in human-computer interaction usually assumes either a single user or a group of users collaborating in the same virtual space each in control of a personal cursor, this project will advance a new model in which a diverse and dynamic group collectively acts as a single operator). These applications will inform a common model for closed-loop crowd support, and will be iteratively improved and evaluated in lab studies and field deployments with blind and deaf users.
Broader Impacts: Project outcomes will enable people with disabilities to overcome more accessibility problems independently and on demand. The PI will release the software tools developed as part of this research as open source code, thereby enabling other researchers to build on his results. Although initially intended for use by blind and deaf people, the tools will likely prove useful as well to people with other disabilities such as cognitive or motor impairments, and people without disabilities may want to outsource interactive tasks which they cannot do, do not want to do, or think the crowd may do better. The PI will conduct annual summer programs in which blind and deaf high school students will develop tools for closed-loop crowd support, and also serve as supporters for people with different disabilities; these activities may encourage some of the participants with disabilities to pursue careers in computing, and the undergraduate students who help develop and run these activities will gain personal exposure to accessible computing and the challenges people with disabilities face in computing that they will carry with them into their careers.
