Co-PIs: Gary Bishop and Dinesh Manocha Department of Computer Science University of North Carolina at Chapel Hill
An auditory display utilizes sound to communicate information to a user and offers an alternative means of visualization. By harnessing the sense of hearing, audio rendering can further enhance a user's experience in a multimodal virtual world. Acoustic realism has many areas of applicability including virtual reality, computer gaming, training systems, desktop interfaces, education, and scientific visualization.
We are conducting an ambitious research program to develop interactive auditory displays. Our goal is to develop new algorithms for physics-based sound synthesis and sound propagation for interactive applications including computer gaming, training systems, and enabling technologies. The approach involves the fusion of both geometry (for high frequencies) and physics (for low frequencies) to model sound propagation and the development of techniques for acoustic levels of detail. To this end we are developing efficient numerical algorithms based on domain decomposition and exploiting modern architecture features to further accelerate the overall performance. We are also evaluating the performance of our algorithms on different applications. In addition to acoustic simulation, our research is generating a fundamental scientific foundation and interactive performance methods for solving wave/sound propagation problems in highly complex domains that span many scientific and engineering disciplines.
Extending the frontier of visual computing, an auditory display utilizes sound to communicate information to a user and offers an alternative means of visualization. By harnessing the sense of hearing, audio rendering can further enhance a user’s experience in a multimodal virtual world. In addition to immersive environments, auditory displays can provide a natural and intuitive human-computer interface for many desktop applications. Furthermore, audio interfaces are increasingly used to develop assistive technologies for the visually impaired. Despite the significance of hearing as one of the dominant senses, auditory displays have not received as much attention as computer graphics. The driving impetus of our research comes from interactive applications, including emergency personnel training systems, desktop interfaces, education, scientific visualization and computer-aided design. Furthermore, interactive modeling and simulation of acoustic spaces can significantly enhance numerous scientific and engineering applications. However, due to intrinsic characteristics of sound, interactive auditory displays pose major computational challenges. INTELLECTUAL MERIT The following major scientific contributions have been achieved during this project: (1) new interactive acoustic algorithms for dynamic virtual environments using acoustics transfer operators, equivalent source methods, and adaptive spatial decomposition; (2) novel sound generation algorithm for liquids based on bubble acoustics; (3) automatic extraction of material properties from one audio sample; (4) innovative application for rapid acoustic prototyping of complex 3D structures for conceptual design and assistive technology. BROADER IMPACT Applications and impacts of interactive auditory displays enabled by new scientific advances include enabling technologies for visually impaired, multimodal human-centric interfaces, immersive teleconferencing, rapid prototyping of acoustic spaces for urban planning, structural design, and reduction of noise pollution. In addition, this research could also offer a fundamentally new solver for wave propagation problems in highly complex, vast domains for seismology, geophysics, meteorology, engineering design, urban planning, etc. The resulting research has been demonstrated to thousands of K-12 students and senior members in the nearby communities and in NC, helping to attract K-12 students and enhance study opportunities for under-represented groups and women students. Two Ph.D. students have completed their doctoral dissertation supported by this grant; and other graduate and undergraduate students working on the project were also partially supported. Resulting software systems have been licensed to a start-up company, creating new jobs for the local community. They are also under further development for designing accessible games for children with disabilities. Tens of refereed publications are disseminated through websites, courses, and international conferences.
