To survive and flourish, people must interact with their environment in an organized fashion. To do so, they need to learn, imagine, and perform an assortment of transformations on and in the world. Primary among these are manipulation of objects and navigation in space. This project integrates research in computer science and cognitive science to develop and evaluate augmented reality tools to create effective dynamic explanations that enhance manipulation and navigation, in conjunction with identification and visualization. Augmented reality refers to user interfaces in which virtual material is integrated with and overlaid on the user?s experience of the real world; for example, by using tracked head-worn and hand-held displays. Dynamic explanations are task-appropriate sequences of actions, presented interactively, with appropriate added information. The tools will be created in collaboration with subject matter experts for exploratory use in indoor and outdoor real world domains: navigating and identifying landmarks in a wooded park area, assembling a piece of furniture, and navigating and visualizing for planning the site of a new urban campus. Cognitive science research will determine the best ways to convey explanations and information to people. Computer science research will address the design and implementation of systems that embody the best candidate approaches for identifying objects and locations, specifying actions, and adding non-visible information. In situ experiments will be used to assess and refine the systems.
Manipulation, navigation, identification, and visualization are representative of important things that people do every day, ranging from fixing broken equipment to reaching a desired destination in an unfamiliar environment. The ways in which we perform these tasks could potentially be improved significantly through augmented reality systems designed using the principles to be developed by this project. Both the cognitive principles and the augmented reality tools will have broad applicability. The systems developed will inform the design of future systems that can aid the general public, for educational and recreational ends, as well as systems that can assist people with auditory, visual, or physical impairments.
