Western tonal music is based on major and minor keys. When people listen to Western music over many years, they acquire an implicit understanding of how the keys are related to each other. This allows listeners to detect "wrong notes" or to perceive a sense of tension in the music. The relationships of the keys to one another, something called tonal space, can be described by their relative position on a torus (a donut-shaped geometrical space). The model developed in this proposal provides quantitative descriptions of how music moves on the torus of tonal space and how our expectations concerning that movement effect our behavioral, emotional, and neural responses. When the model predicts that a particular note or chord is unexpected, will it be perceived as an unexpected or "wrong' note? The proposed work will also explore the timescales over which tonal expectations are structured. Is the model-predicted brain activity that follows music's movements in tonal space the same as the brain circuitry actually engaged when memories are evoked by music?
The broader significance of this research lies in relating structural aspects of music to specific psychological and neural processes that underlie our ability to perceive, appreciate, and respond to music. The model that is being developed is centered on the concept of expectation. Whether the sensory input that the world provides matches our expectations is emerging as a general principle for how our brains function. By examining how expectations operate in the brain in association with music, an extremely powerful and compelling source of stimulation for humans, the investigators hope to gain more general insight into the functional organization of the human brain.
The primary objectives of this project were (1) to examine the ability of a computational model to explain behavioral responses to expectancy violating tonal events, e.g. "wrong notes" or "wrong chords," and (2) to make the computer programs underlying this model publically available. The model transforms an audio signal through several stages of processing intended to represent neural and psychological mechanisms. Because it starts with the raw audio signal, the model is faced with a more difficult task when trying to explain aspects of high-level music cognition than are models that begin at a higher stage of representing musical notes. We analyzed response time data, i.e. how long it takes a person to make a judgment about a target event that follows a context (melody or chord progression), from seven experiments that were designed to manipulate the degree to which participants' responses could be determined by the sensory information present in the contexts ("sensory processing") versus determined by the activation of musical knowledge about major and minor keys stored in long-term memory ("cognitive processing"). We used the ouput of different stages of the model to represent more "sensory" and more "cognitive" levels of processing, and combined these outputs in a statistical model in order to determine the relative contributions of each processing stage to predicting the reaction times that were observed for each of the stimuli used in the different studies (over 300 different stimuli). In the course of our work we discovered that we had to add a stage of processing that models the effect of "closure" in a melody or chord progression, i.e. makes the musical sequence sound complete, in order to successfully model the reaction times from all of the experiments. Our model performed better than all previous models that could be used to quantitatively model the relationship between the tonal properties of specific musical stimuli and behavioral responses. As such is provides an important tool for analyzing musical audio in terms of known neural and psychological responses to music. These analyses in turn allow for analyses of behavioral data and neuroimaging data so that the scientific community can better understand how we perceive music, how it is stored in long term memory, and how it interacts with other aspects of higher cognition, such as autobiographical memories.
