The process by which humans (and other animals) perceive space from auditory input is complex. Over the past decades, researchers worldwide (including those involved in the current proposal) have elucidated the mechanisms by which humans perceive sound and are able to develop algorithms for the creation of immersive audio. These include audio from games, from the mixing of music, and from the capture of live events. Because the wavelengths of audible sound are comparable to the sizes of environmental objects and of features on the listeners' bodies, the sound perceived by the listener is "colored" by scattering off these objects. Understanding and efficiently approximating the room impulse response and the Head Related Transfer Function (HRTF), which respectively characterize these scattering processes, has been a major contribution of the PI's research over the past decade. As each listener?s physical body features have different shapes and sizes, the HRTF shows considerable inter-personal variation. The PI has developed extremely rapid techniques for measuring HRTFs, for characterizing HRTFs based on the body and ear measurements of listeners, and for rapid creation of immersive sound using HRTFs and room responses. Further, single microphone recordings lose the spatial information in the sound signal. To retain the spatial information during auditory environment capture, the PI has developed novel spherical microphone array technology. A decomposition of the captured sound in terms of plane-wave basis functions allows the easy incorporation of HRTFs in the captured sound.
Today's consumers enjoy music, games, and other media on their mobile devices; most settle for lackluster sound produced over headphones. Current headphone sound is simply unable to produce the engaging sound experience that high-end music systems, movie theaters, or live events deliver. With the technologies developed under previous NSF support, the team has allowed entertainment content creators to achieve unmatched realism in sound presentation over headphones. Combined with the team's sound capture hardware, immersive reproduction of live events can be achieved. In the last year, the number of smartphone sales outpaced that of PCs for the first time in history; consequentially, the amount of content consumed over headphones is vastly increasing. Our technology will allow companies to make headphone listening more immersive and reach that growing consumer base. Several industries produce and consume audio products. These include mobile device / gaming / PC hardware manufacturers, entertainment content producers, and content delivery networks. They will all be positively impacted by the technology. Because of the large market, and the strength of the team, they anticipate that a large and successful business can be built around this technology. If successful, this effort will contribute to ongoing U.S. leadership in the fields of mobile technologies, gaming, entertainment, and immersive simulation.
NSF funded research had helped University of Maryland researchers create an understanding of how humans perceive sound in three dimensions. This research was thought to have significant commercial application in domains like gaming, music, cinema, user interfaces for visually impaired, and data presentation via sonification. The NSF ICorps project was a structured educational program and practicum to educate the university researchers (PI and graduate student), working with an external mentor, as well as NSF appointed instructors, to help the inventors scientifically determine if a viable business was possible based around the technology. Over one hundred and fifty customers were interviewed following a structured approach taught to the team by ICorps instructors. A "minimum viable product" demonstration in gaming was created. Based on the results of these interviews, VisiSonics corporation (co-founded by the university researchers) licensed the relevant IP from the University, and sought to develop the product. The company is initially targeting markets in gaming and virtual reality, and later intends to address the other markets. It has applied for NSF Small Business Innovation Research Funding. A software library, based on the University IP, that can be incorporated into gaming engines and digital media players is being developed, and is under testing by various entities.
