Project Proposed: This project, developing an integrated virtual environment system capable of allowing not only 3D visualization of data, but also interaction with data through natural hand and finger gestured based on a dual interface, exploits a multi-touch interaction interface and a vision based hand-gesture interface. Virtual reality environments rely on a collection of technologies that allow the user to go through a coherent and unified perceptual experience involving multiple senses, such as vision, touch, and sound, while interacting with 3-dimensional data. These immersive, highly visual, 3D environments currently offer a fairly high level of performance for spatially visualizing data. However, the corresponding machinery providing user interaction with these systems has not kept up the same pace of development with the visualization tools. At present some advanced commercial environments offer some user interaction capabilities achieved through wired wearable hardware (such as wired gloves and head mounted displays). This promotes, in turn, an unnatural and cumbersome interaction between the user and the virtual reality environments, curbing the acceptance of the technologies. The syntax and semantics of the hand and finger gestures developed interacts with geometric data, while the implementation relies on the Multi-Touch Surface computing platform as well as on a newly developed gesture tracking and recognition system. The environment should lead to a potent open platform for interacting with virtual geometric data in an intuitive way, without the need for wearable hardware galvanizing the state of the art at the institution in Nano and Design Engineering, Psychology, Computer Science, Structural biology, as well as support research in the Center for Health, Intervention, and Prevention (CHIP). This project addresses the problem. Broader Impacts: This instrumentation will be open source and widely available with well documented set up procedures. The VR system will be networked with other VR-sites, including VRAC at Iowa State, to maximize the impact and stimulate technology transfer. Moreover, the instrument contributes to - Stimulate critical avenues of interdisciplinary research involving engineering, biology, computer science, psychology, and Human Computer Interaction (HCI), - Strengthen the potential for educational, student recruiting, and outreach activities, and - Perform targeted outreach to K-12 students, teachers, and school districts serving groups that have traditionally been underrepresented in the engineering disciplines. The project also advances the state of the art in the teaching and practice of engineering design, as well as other fields in which geometry plays and important role. Contributing to the development of a new generation of professionals that use capabilities of virtual reality tools to augment traditional disciplines for improved engineering design, this work should have a long-lasting impact on the ability of scientists and engineers.
