The visual cortex of the rhesus monkey can be divided into several functional areas. Areas V1, V2, V4 and IT each have a crucial role in visual form and color processing. One representative visual task is short-term visual memory, which is used to compare a picture shown now with one shown previously. This is called a delayed-matching-to-sample, or DMS, task. Lesions in area TE of IT cortex interfere with DMS performance. Thus, it is reasonable to assume that TE neurons play some role in DMS, but what might that be? Success in this simple visual memory task requires coding, memory, recall, comparison and decision steps. TE neurons are selective for visual features, and so a role in image encoding has been assumed. Here we examine the possibility that TE neurons may actually hold a short-term memory trace during DMS tasks. We recorded activities of 35 TE neurons from two rhesus monkeys. Eight black and white patterns were used as test stimuli. Responses of neurons within TE fluctuated considerably across repeated presentations of a single stimulus. We measured the correlations between trial-by-trial fluctuations in different task phases (sample, nonmatch, and match). The noise on the sample and match responses correlated more strongly than sample and nonmatch fluctuations, even though the interval between sample and nonmatch was shorter than the interval between sample and match (median variance explained: sample vs. match = 14%; sample vs. nonmatch = 9%). Such trial-by-trial correlation between sample and match noise is strong evidence for local storage of the short-term memory trace. These data lead us to propose a new theory of the role of TE neurons in DMS tasks, in which the TE neurons actually store the short-term iconic memory trace in their synaptic weights.? The average activities of TE neurons are selective for images. However, they also showed a very surprising result: the deviations of the sample responses correlated more strongly with the match than nonmatch deviations. There seems to be no way to explain these correlations if the TE neurons only encode the stimulus, because noise in encoding must be independent across different stimulus presentations, and thus can not preserve fluctuations across events. Here we propose a new hypothesis of iconic, short-term memory: these TE neurons hold the memory trace of the sample image in the strength of its synaptic inputs using a form of one-trial-learning. This population of neurons thus forms a matched filter (the best filter for detecting the presence of a known signal in white noise) for the sample image. The power in the responses of the population of TE neurons, but not in individual neurons, is higher for the match than for the nonmatch stimulus. This new theory is consistent with an idea we proposed in 1992 (Eskandar et al.), when we showed that responses of IT cortex neurons contained information about the sample and current images in a DMS task. Then, we fitted the responses of the neuron with a model that multiplied the encoding of the sample stimulus, recalled from a memory store, by the encoding of the current stimulus. This measured the correlation between the two images and formed the basis for a decision by a higher center. The new theory also uses a multiplicative model, but based on the evidence in the in this study posits that the memory trace is stored in the TE neurons themselves. This greatly simplifies the structure of the model needed to perform the DMS task, because the memory store, recall and correlation are all combined in the functions of one neuronal population.? The matched filter model has no free parameters. However, the model is tuned for each monkey by setting the noise input levels so that the success of the model on the DMS task matches the success of the monkey. With this noise setting, the model also closely matches the correlated noise found in the monkey. Note that this noise is not part of the memory process. It is not necessary to perform the DMS task, and thus provides no evolutionary advantage to the animal. The correlated noise on the sample and test stimulus responses is thus an epiphenomenon that acts as a signature for probing the mechanism of short-term memory.
