The goal of this project is to gain a fundamental quantitative understanding of the mechanisms of visual short-term memory (VSTM) in health. VSTM is a central aspect of human cognition, allowing people to detect changes in the environment. Deficits in VSTM are found in many visual and neurological disorders, including visual neglect, parietal and frontal lobe damage, attention deficit/hyperactivity disorder, traumatic brain injury, and schizophrenia. A better characterization of VSTM may lead the way to better diagnosis and treatment of such deficits. In its first period, this project has overturned established beliefs about the nature of the limitations of VSTM. The continuation of the project focuses on the richness of VSTM and in particular the hypothesis that VSTM contains a representation of uncertainty associated with a short-term memory. To test this hypothesis, the investigators use a set of novel behavioral tasks, some of which involve subjects directly reporting their confidence about a memory, while others are designed such that decision performance benefits from utilizing knowledge of VSTM uncertainty. In contrast to other work that uses stimuli such as line drawings or letters, all experiments in this project use simple, single-feature stimuli, allowing for both tight experimental control and precise mathematical modeling. The project integrates mathematical modeling with the experiments in an essential way. The Ma laboratory is a leader in the large-scale testing of mathematical models of behavior, comparing them using state-of-the-art statistical techniques, and using the results to inform further experimental design. The models in this project are aimed at precisely characterizing the processes by which the brain arrives at a VSTM-based decision. One large category of models that will be tested is based on the notion that observers optimally or near-optimally combine VSTM information with prior and reward information to reach a decision. Beyond behavioral models, the project will use neural network modeling to address how neural circuits might implement the computations involved in the behavioral tasks of the project. Specifically, the investigators ask how the brain can incorporate knowledge of VSTM uncertainty without being explicitly trained on the correct value of uncertainty. The broader impact of the project consists of connecting two traditionally disparate research areas, namely the study of VSTM and the study of decision-making. Besides potential clinical applications, the project will lead to a better basic understanding of how the brain manages to make good decisions in the face of great uncertainty.

Public Health Relevance

Deficits in visual short-term memory are found in many forms of brain damage and disease, including visual neglect, parietal and frontal lesions, attention deficit/hyperactivity disorder, and Alzheimer's disease. Here, we propose to better characterize, through experiment and theory, the behavioral and neural mechanisms underlying visual short-term memory, with the eventual goal of improving the diagnosis and treatment of these disorders.

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Cognition and Perception Study Section (CP)
Program Officer
Wiggs, Cheri
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
New York University
Schools of Arts and Sciences
New York
United States
Zip Code
Acerbi, Luigi; Dokka, Kalpana; Angelaki, Dora E et al. (2018) Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception. PLoS Comput Biol 14:e1006110
van den Berg, Ronald; Ma, Wei Ji (2018) A resource-rational theory of set size effects in human visual working memory. Elife 7:
van den Berg, Ronald; Yoo, Aspen H; Ma, Wei Ji (2017) Fechner's law in metacognition: A quantitative model of visual working memory confidence. Psychol Rev 124:197-214
Shin, Hongsup; Ma, Wei Ji (2017) Visual short-term memory for oriented, colored objects. J Vis 17:12
Mihali, Andra; van Opheusden, Bas; Ma, Wei Ji (2017) Bayesian microsaccade detection. J Vis 17:13
Shin, Hongsup; Zou, Qijia; Ma, Wei Ji (2017) The effects of delay duration on visual working memory for orientation. J Vis 17:10
Orhan, A Emin; Ma, Wei Ji (2017) Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nat Commun 8:138
Bhardwaj, Manisha; van den Berg, Ronald; Ma, Wei Ji et al. (2016) Do People Take Stimulus Correlations into Account in Visual Search? PLoS One 11:e0149402
Shin, Hongsup; Ma, Wei Ji (2016) Crowdsourced single-trial probes of visual working memory for irrelevant features. J Vis 16:10
Shen, Shan; Ma, Wei Ji (2016) A detailed comparison of optimality and simplicity in perceptual decision making. Psychol Rev 123:452-80

Showing the most recent 10 out of 29 publications