The PI introduces the term Anywhere Augmentation to refer to the idea of linking location-specific computing services with the physical world, making them readily and directly available in any situation and location. This project embodies a novel approach to Anywhere Augmentation based on efficient human input for wearable computing and augmented reality (AR), through both sketch-based interfaces on hand-held devices and direct-overlay 3D user interfaces. Mobile augmented reality is a powerful interface for wearable computing. If computer users are enabled to place arbitrary annotations in 3D space wherever they go, the physical world becomes the user interface. Instead of embedding computing and display equipment in the environment as in the case of ubiquitous computing, graphical annotations are overlaid on top of the environment by means of optical see-through glasses or video overlay. Robust registration between the physical world and the augmentations is necessary. Current approaches rely on the availability of a 3D model of the environment or on its instrumentation with active or passive markers. The PI's approach is novel in that he proposes to emphasize, support, and utilize the expertise of the human in the loop to make Anywhere Augmentation feasible. Human users generally have a clear grasp of the layout of the scene in front of them, and they can easily identify the physical objects with which information should be linked. The PI plans to enable users to transfer their intuitional scene understanding to the computer through intelligently constrained and assisted user interfaces. Current real-time computer vision techniques and algorithms, while far from being able to facilitate automatic scene understanding, are very well suited to constrain and guide a user?s informed input for scene analysis and augmentation, delivered in the form of a few simple point selections, stroke gestures, and common classifications. As a main source of input, the PI will employ video feeds from small head-worn or palm-top-device cameras. The devices, algorithms and interaction techniques will be applicable to novel settings and application scenarios (e.g., visualization of occluded infrastructure, navigational guidance, and social and educational applications for high-school students).
Broader Impact: The PI will use this research as a case study and platform for projects supporting the teaching of human-computer interaction fundamentals. As a first step towards actively furthering the inclusion of minority students in the benefits of the research, the PI has partnered with Jackson State University, MS, and will also collaborate with outreach programs supporting local underrepresented K-12 students, where he will enhance the after-school programs and field trips with carefully planned AR experimentation and mentoring. The goal of Anywhere Augmentation will be tested by making research products (new devices, tools, and interfaces) available to students and field scientists in a wide variety of environments (e.g., as a campus navigation and inventory tool, for an emergency response scenario, at UCSB lab open houses, and at international conferences). The PI will also introduce innovations in three courses in the curriculum of the UCSB Computer Science Department (an undergraduate elective on HCI which he established, the undergraduate senior CS design project course cycle, and a new graduate course on 3D user interfaces), so they include hands-on experiences on effective UI design and programming for mobile devices, including mobile AR interfaces. To this end, the PI will also involve the UCSB Allosphere, a 3-story spherical surround-view 3D immersive space, as a mobile AR simulator.
