The visual cortex of the rhesus monkey can be divided into several functional areas. Areas V1, V2, V4 and IT each have a crucial role in visual form and color processing. One representative visual task is short-term visual memory, which is used to compare a picture shown now with one shown previously. This is called a delayed-matching-to-sample, or DMS, task. Lesions in area TE of IT cortex interfere with DMS performance. Thus, it is reasonable to assume that TE neurons play some role in DMS, but what might that be? Success in this simple visual memory task requires coding, memory, recall, comparison and decision steps. TE neurons are selective for visual features, and so a role in image encoding has been assumed. Here we examine the possibility that TE neurons may actually hold a short-term memory trace during DMS tasks. We recorded activities of 35 TE neurons from two rhesus monkeys. Eight black and white patterns were used as test stimuli. Responses of neurons within TE fluctuated considerably across repeated presentations of a single stimulus. We measured the correlations between trial-by-trial fluctuations in different task phases (sample, nonmatch, and match). The noise on the sample and match responses correlated more strongly than sample and nonmatch fluctuations, even though the interval between sample and nonmatch was shorter than the interval between sample and match (median variance explained: sample vs. match = 14%;sample vs. nonmatch = 9%). Such trial-by-trial correlation between sample and match noise is strong evidence for local storage of the short-term memory trace. These data lead us to propose a new theory of the role of TE neurons in DMS tasks, in which the TE neurons actually store the short-term iconic memory trace in their synaptic weights. The average activities of TE neurons are selective for images. However, they also showed a very surprising result: the deviations of the sample responses correlated more strongly with the match than nonmatch deviations. There seems to be no way to explain these correlations if the TE neurons only encode the stimulus, because noise in encoding must be independent across different stimulus presentations, and thus can not preserve fluctuations across events. Here we propose a new hypothesis of iconic, short-term memory: these TE neurons hold the memory trace of the sample image in the strength of its synaptic inputs using a form of one-trial-learning. This population of neurons thus forms a matched filter (the best filter for detecting the presence of a known signal in white noise) for the sample image. The power in the responses of the population of TE neurons, but not in individual neurons, is higher for the match than for the nonmatch stimulus. This new theory is consistent with an idea we proposed in 1992 (Eskandar et al.), when we showed that responses of IT cortex neurons contained information about the sample and current images in a DMS task. Then, we fitted the responses of the neuron with a model that multiplied the encoding of the sample stimulus, recalled from a memory store, by the encoding of the current stimulus. This measured the correlation between the two images and formed the basis for a decision by a higher center. The new theory also uses a multiplicative model, but based on the evidence in the in this study posits that the memory trace is stored in the TE neurons themselves. This greatly simplifies the structure of the model needed to perform the DMS task, because the memory store, recall and correlation are all combined in the functions of one neuronal population. The matched filter model has no free parameters. However, the model is tuned for each monkey by setting the noise input levels so that the success of the model on the DMS task matches the success of the monkey. With this noise setting, the model also closely matches the correlated noise found in the monkey. Note that this noise is not part of the memory process. It is not necessary to perform the DMS task, and thus provides no evolutionary advantage to the animal. The correlated noise on the sample and test stimulus responses is thus an epiphenomenon that acts as a signature for probing the mechanism of short-term memory. This model does not specify how a neuronal circuit might perform matched filtering. Cortical architecture, connections and dynamics are all ignored. Nonetheless, our model does more than merely provide a representation of the data. Our data show only that cells are selective for stimuli, respond differently depending upon the condition (sample, nonmatch, match), and that there is correlated noise on their responses across sample-test presentations. The matched-filter model demonstrates (thus, it is an existence proof) that the responses themselves (not the noise) could come from a multiplicative interaction between a synaptic memory trace and a current input. This model is a very strong argument in favor of the matched-filter theory of short-term memory, even without a biophysically detailed model. Furthermore, the matched-filter hypothesis explains why the noise correlation exists, even though it makes no contribution to solving the memory task. It is important to remember that the noise correlations are not built into the model by fitting it to the data (e.g, perirhinal neurons are fit the same way but have no such correlations). Thus, this black-box model, although simplistic with respect to cortical circuitry, provides a mechanistic explanation of how the noise correlations could arise, and thus is a plausible model for short-term memory. We conclude that area TE neurons store a synaptic memory trace during short-term visual memory.
| Sugase-Miyamoto, Yasuko; Liu, Zheng; Wiener, Matthew C et al. (2008) Short-term memory trace in rapidly adapting synapses of inferior temporal cortex. PLoS Comput Biol 4:e1000073 |
