"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."
Music-making is universal and has been a conduit for human creativity for at least tens of thousands of years, and music plays an essential role in human social bonding, emotional communication, and entertainment. Because information technology and entertainment constitute an increasing share of the total economic output in developed countries, creativity lies at the heart of the modern economy. Yet the creativity that underlies musical domains is poorly understood. In this project the PI and his team will seek to understand, model, and support improvisation, or real-time collaborative creativity, in the context of music. Most musical traditions in the world use improvisation as a method of creativity, so analysis and modeling of improvisation in highly evolved musical systems should provide essential insights into creative activity. This study will consider a representative subset of musical traditions, in order to keep the results as broadly applicable as possible: jazz, Indian classical music, and avant-garde art music. The research will employ an interdisciplinary approach involving ethnography, music theory, statistical modeling, machine learning, signal processing and instrument design, and cognitive studies. The objectives are to develop computational models of improvisation and to use them to develop new technologies that support creativity in music and education. To these ends, the PI and his team will investigate machine modeling of musical improvisation as a means to understand real-time creativity. They will perform analysis of improvisational traditions from many cultures and attempt to unify them using a common probabilistic framework. They will also conduct systematic evaluation of formal models in realistic performance contexts, and use brain imaging of improvising musicians to gain insight into highly creative mental activity. Ethnography will be applied to characterize the diversity of improvisational traditions, while music theory will help define basic concepts and provide working hypotheses and frameworks for formal models. Relationships between musical concepts will be represented using a probabilistic, generative model that seeks to represent the complex conditional dependencies among musical parameters as well as among performers; this abstraction will help unify different surface traditions within a common framework, revealing essential patterns in systems for real-time creativity. Through modeling, the study will explore how improvising musicians learn these systems and reference them in performance. Cognitive studies will elucidate whether a distinct set of cognitive processes are employed during improvisation; understanding what neural networks are active may be a first step in engineering systems that foster creativity in other contexts. Taken together, these studies will improve our understanding of creative processes.
Broader Impacts: Studying the arts, where creativity has been developed, refined, and systematized, will offer insight into developing systems that foster creativity in other areas such as engineering and information technology. Improvisation systems built from this research will be used to create ensembles through which programming, computational modeling, and creative problem solving may be taught. The development of technology to support improvisation will also help teach creativity alongside rote performance skills, and encourage new programming paradigms. For artists, project outcomes will enable new improvisation systems to be built using reconfigurable building blocks, thereby catalyzing research in improvisation and the development of sophisticated systems. Planned collaborations with world-renowned musicians will expose a broad public to this research, and will help recruit new students to engineering, turn public attention to the study of creativity, and hopefully also create satisfying works of art.
