The goal of this work is to gain a fundamental, quantitative understanding of the mechanisms of visual short- term memory (VSTM) in health. Deficits in VSTM are found in numerous disorders, including visual neglect, parietal and frontal lobe damage, attention deficit/hyperactivity disorder, and schizophrenia. A better characterization of VSTM loss may point the direction of therapy tasks to help restore some of the loss. This research proposal relies in an essential way on integrating psychophysics with neuroscience. The leading class of VSTM models asserts that VSTM is a noiseless storage with a fixed, limited capacity of about 4 items. Extra items, if any, will not be remembered. We propose an alternative theory that casts the limitations of VSTM in terms of the neural mechanisms of low-level vision. Sensory information comes with uncertainty, in part due to neural variability. In simple perceptual tasks like cue combination, it is well known that humans perform probabilistic inference to optimize performance under such uncertainty. Applying the same concepts to VSTM, we postulate that: 1) uncertainty increases with set size due to a neural constraint;2) the brain performs probabilistic inference on uncertain inputs. We call this the uncertainty model.
Aim 1 : To test whether the uncertainty model or fixed-capacity models better explain delayed estimation performance. Subjects estimate the identity of a remembered item. We will use two distinct tasks to measure subjects'uncertainty as a function of set size. We will test the hypothesis that VSTM is limited not by a fixed capacity, but by a constraint on neural resources which are distributed continuously among items.
Aim 2 : To test whether the uncertainty model or fixed-capacity models better explain change detection performance. Change detection is a leading paradigm for studying VSTM. We will test the hypothesis that observers optimally detect changes under uncertainty by computing the probability of a change given the noisy observations (probabilistic inference), in analogy to low-level visual tasks.
Aim 3 : To test the hypothesis that human observers optimally integrate likelihoods and priors in change detection. An optimal observer uses knowledge of uncertainty on an item-to-item and trial-to-trial basis in downstream computation. To test whether humans do this in change detection, we will vary either the likelihood or a prior, at fixed set size, by manipulating contrast and overall task statistics, respectively.
Aim 4 : To model the neural basis of visual change detection. Informed by the experimental findings in Aims 1-3, we will construct a behaviorally constrained neural network for change detection. We will use the theoretical framework of probabilistic population coding. The resulting network will be entirely based on the uncertainty model but exhibit the appearance of a capacity limit. It will serve as a basis for physiological tests.

Public Health Relevance

Deficits in visual short-term memory are found in many forms of brain damage and disease, including visual neglect, parietal and frontal lesions, attention deficit/hyperactivity disorder, and Alzheimer's disease. Here, we propose to better characterize, through experiment and theory, the behavioral and neural mechanisms underlying visual short-term memory, with the eventual goal of improving the diagnosis and treatment of these disorders.

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Central Visual Processing Study Section (CVP)
Program Officer
Steinmetz, Michael A
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Baylor College of Medicine
Schools of Medicine
United States
Zip Code
Orhan, A Emin; Ma, Wei Ji (2017) Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nat Commun 8:138
van den Berg, Ronald; Yoo, Aspen H; Ma, Wei Ji (2017) Fechner's law in metacognition: A quantitative model of visual working memory confidence. Psychol Rev 124:197-214
Mihali, Andra; van Opheusden, Bas; Ma, Wei Ji (2017) Bayesian microsaccade detection. J Vis 17:13
Shin, Hongsup; Ma, Wei Ji (2016) Crowdsourced single-trial probes of visual working memory for irrelevant features. J Vis 16:10
Bhardwaj, Manisha; van den Berg, Ronald; Ma, Wei Ji et al. (2016) Do People Take Stimulus Correlations into Account in Visual Search? PLoS One 11:e0149402
Shen, Shan; Ma, Wei Ji (2016) A detailed comparison of optimality and simplicity in perceptual decision making. Psychol Rev 123:452-80
Orhan, A Emin; Ma, Wei Ji (2015) Neural population coding of multiple stimuli. J Neurosci 35:3825-41
Ma, Wei Ji; Shen, Shan; Dziugaite, Gintare et al. (2015) Requiem for the max rule? Vision Res 116:179-93
Devkar, Deepna T; Wright, Anthony A; Ma, Wei Ji (2015) The same type of visual working memory limitations in humans and monkeys. J Vis 15:13
van Bergen, Ruben S; Ma, Wei Ji; Pratte, Michael S et al. (2015) Sensory uncertainty decoded from visual cortex predicts behavior. Nat Neurosci 18:1728-30

Showing the most recent 10 out of 25 publications