This Small Business Innovation Research (SBIR) Phase I project aims at developing a computational music system that will automatically analyze, re-synthesize and distribute digital music with the goal of transforming a linear and passive music listening practice into an interactive, expressive and viral music performance experience. The system will utilize approaches for temporal, tonal and harmonic analysis as well as collaborative interactive performance and distribution, in an effort to create an engaging musical experience that is rewarding and viral. Mobile and web-based applications will be developed supporting cloud-based multi-player interactions, allowing music fans to get expressively involved in creating, modifying, and personalizing their favorite music. The project will investigate how temporal structures in music are represented and processed by human listeners and will design new models for analysis and re-synthesis of music in an effort to facilitate interactive and collaborative musical experiences. The project will advance current knowledge in areas such as music information retrieval, music theory, music perception, machine learning, mobile interaction, signal processing, instrument design, and cloud computing.
If successful, the project will lead to broad impact in the public sphere by creating engaging and rewarding musical experience for users at all skill levels. By providing easy-to use tools for music creation and sharing, the project would encourage creativity and expression for the general population, facilitating growth in user-generated content in an area that is usually not accessible to the masses. From a business perspective, the broad impact of the project will be in providing a novel solution to the significant problems faced by the music industry today. The industry is looking for new ways to monetize content by engaging fans with music games and personalization tools (e.g. ring tones). Once deployed, the platform proposed will address these needs by providing an intelligent system that will allow fans to interact, personalize and share their favorite music in an expressive and viral manner, using mobile devices and online.
The innovative aspect of the research is the automation of Music Information Retrieval processes based on human musical perception for the purpose of re-synthesis of music in an interactive and engaging format. The quantitative information we extracted from a musical analysis has been used to automatically create interactive musical experience through the detection and transformation of musical elements such as beat, bars, tempo, segments, scale, and gestural mapping. These data have enabled the creation of an application that transforms a smartphone into an interactive, educational and expressive music performance tool. The intellectual merit of the project represents its fundamental contribution to human knowledge in music perception and cognition, performance analysis, and collaborative music interaction. We investigated how temporal structures in music are represented and processed by human listeners and designed new models for analysis, and re-synthesis of music in an effort to facilitate expressive and creative collaborative interaction. During the course of the Phase I period of the NSF-SBIR project, we developed the "ZOOZifier" – an OS/X application that expedites the process of creating sample-based instruments and loops that enabled our flagship product, ZOOZbeat. ZOOZBeat was designed to give the average non-musical consumer the ability to create music by turning the average smartphone into a mobile gesture-based instrument and recording studio. By utilizing the accelerometers on a smartphones we allow the user to choose an instrument and add their creative expression to create a new musical composition. By simplifying the process of creating compositions we allowed everyone from children to accomplished musicians an engaging way to express themselves musically right from their mobile device and, thus, to experience the thrill of musical composition. As part of our partnership with the Coca Cola Company in developing the ZOOZbeat Sprite product, we were awarded Branded Music App of the Year by Billboard Magazine. Our mobile application received rave reviews from publications such as Tech Crunch, Wired, CNN; reached top 10 applications from Apple's App Store; and was downloaded over 2 million times.
