The immersive graphics, the large amount of user-generated content, and the social interaction opportunities offered by the greater sophistication of virtual worlds are leading to an explosive growth in their popularity, with millions of participating users. Unfortunately, however, virtual worlds are not accessible to users who are visually impaired. In this project the PI's goal is to alleviate this deficiency by developing a virtual world interface for visually impaired users called TextSL, which is able to extract a textual representation from a virtual world that can be read with a screen reader. Users interact with TextSL by means of a command-based mechanism inspired by multi-user dungeon games. A prototype of the system that allows for a basic form of access to the popular virtual world of Second Life has shown that such an approach is feasible. But the prototype lacks two of the most important features offered by virtual worlds: the ability to create content, and the ability to interact with interactive objects. Additional problems identified by the PI's preliminary work are that less than half of the objects in Second Life have a meaningful descriptive name, which makes the majority of objects invisible to visually impaired users, and that virtual worlds contain such vast amounts of content that merely converting it somehow into a textual description may easily overwhelm the user. To address such challenges, the PI will develop a game based on the "games-with-a-purpose" paradigm that allows sighted users to make virtual worlds accessible to visually impaired users by labeling objects without a name, while also establishing a taxonomy of objects that can be used to provide a more usable form of feedback. He will create a new 3D object recognition algorithm based on the "bag-of-features" approach that takes advantage of the unique way objects are represented in Second Life, and which can efficiently and automatically recognize large numbers of objects without a name. And he will also develop techniques that are able to provide feedback on a user's environment without overwhelming the user with large amounts of information, that are able to convey textually a user's interaction with an interactive object, and that support creation of 3D objects using command or haptic based methods.
Broader Impacts: This research will increase the quality of life for the millions of people who are visually impaired, by allowing them to participate more fully in the information society. In particular, the social interaction offered by virtual worlds will benefit people who are visually impaired as well as other individuals whose disabilities often lead to both physical and social isolation. Furthermore, as virtual worlds are increasingly used as educational platforms, access for all to this technology is doubly critical. The interaction models of virtual worlds resemble those of games, so project outcomes may also provide new insights into how to make games accessible to the visually impaired.
