This project investigates hierarchical machine intelligence for modeling and composing complex soundscapes. We adapt methods for extracting time-dependent information from text documents to the problem of extracting spectral (graphical) patterns and the probabilities that they occur or co-occur in soundscapes. We are analyzing the spectral patterns that emerge from sound files of many types, including recordings of building interiors with regular foot traffic, musical files, and synthesized sound. A significant part of the research is in devising spectral features that are important for this kind of mapping/identification.
Research under this award is also investigating the use of reinforcement learning (RL) to identify time-dependent 'landmarks' from soundscape models, and we employ RL agents to compose large soundscapes from thousands of millisecond length grains of sound in a process called granular synthesis. Systems of RL agents enable us to study distributed time-dependent RL agents in a complex environment, with the ability to produce aural demonstrations of the agents' learning. We also expect the system to produce some compositions that are pleasing in the electronic music sense.
This research will have an impact on curricular efforts in Arts and Technology at Smith College, supporting the Computer Science Department's efforts to attract more students, especially to research.
