The goal of this work is to gain a fundamental, quantitative understanding of the mechanisms of visual short- term memory (VSTM) in health. Deficits in VSTM are found in numerous disorders, including visual neglect, parietal and frontal lobe damage, attention deficit/hyperactivity disorder, and schizophrenia. A better characterization of VSTM loss may point the direction of therapy tasks to help restore some of the loss. This research proposal relies in an essential way on integrating psychophysics with neuroscience. The leading class of VSTM models asserts that VSTM is a noiseless storage with a fixed, limited capacity of about 4 items. Extra items, if any, will not be remembered. We propose an alternative theory that casts the limitations of VSTM in terms of the neural mechanisms of low-level vision. Sensory information comes with uncertainty, in part due to neural variability. In simple perceptual tasks like cue combination, it is well known that humans perform probabilistic inference to optimize performance under such uncertainty. Applying the same concepts to VSTM, we postulate that: 1) uncertainty increases with set size due to a neural constraint;2) the brain performs probabilistic inference on uncertain inputs. We call this the uncertainty model.
Aim 1 : To test whether the uncertainty model or fixed-capacity models better explain delayed estimation performance. Subjects estimate the identity of a remembered item. We will use two distinct tasks to measure subjects'uncertainty as a function of set size. We will test the hypothesis that VSTM is limited not by a fixed capacity, but by a constraint on neural resources which are distributed continuously among items.
Aim 2 : To test whether the uncertainty model or fixed-capacity models better explain change detection performance. Change detection is a leading paradigm for studying VSTM. We will test the hypothesis that observers optimally detect changes under uncertainty by computing the probability of a change given the noisy observations (probabilistic inference), in analogy to low-level visual tasks.
Aim 3 : To test the hypothesis that human observers optimally integrate likelihoods and priors in change detection. An optimal observer uses knowledge of uncertainty on an item-to-item and trial-to-trial basis in downstream computation. To test whether humans do this in change detection, we will vary either the likelihood or a prior, at fixed set size, by manipulating contrast and overall task statistics, respectively.
Aim 4 : To model the neural basis of visual change detection. Informed by the experimental findings in Aims 1-3, we will construct a behaviorally constrained neural network for change detection. We will use the theoretical framework of probabilistic population coding. The resulting network will be entirely based on the uncertainty model but exhibit the appearance of a capacity limit. It will serve as a basis for physiological tests.

Public Health Relevance

Deficits in visual short-term memory are found in many forms of brain damage and disease, including visual neglect, parietal and frontal lesions, attention deficit/hyperactivity disorder, and Alzheimer's disease. Here, we propose to better characterize, through experiment and theory, the behavioral and neural mechanisms underlying visual short-term memory, with the eventual goal of improving the diagnosis and treatment of these disorders.

Agency
National Institute of Health (NIH)
Institute
National Eye Institute (NEI)
Type
Research Project (R01)
Project #
5R01EY020958-03
Application #
8306938
Study Section
Central Visual Processing Study Section (CVP)
Program Officer
Wiggs, Cheri
Project Start
2010-08-01
Project End
2015-07-31
Budget Start
2012-08-01
Budget End
2013-07-31
Support Year
3
Fiscal Year
2012
Total Cost
$294,720
Indirect Cost
$102,720
Name
Baylor College of Medicine
Department
Neurosciences
Type
Schools of Medicine
DUNS #
051113330
City
Houston
State
TX
Country
United States
Zip Code
77030
Acerbi, Luigi; Dokka, Kalpana; Angelaki, Dora E et al. (2018) Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception. PLoS Comput Biol 14:e1006110
van den Berg, Ronald; Ma, Wei Ji (2018) A resource-rational theory of set size effects in human visual working memory. Elife 7:
Orhan, A Emin; Ma, Wei Ji (2017) Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nat Commun 8:138
van den Berg, Ronald; Yoo, Aspen H; Ma, Wei Ji (2017) Fechner's law in metacognition: A quantitative model of visual working memory confidence. Psychol Rev 124:197-214
Shin, Hongsup; Ma, Wei Ji (2017) Visual short-term memory for oriented, colored objects. J Vis 17:12
Mihali, Andra; van Opheusden, Bas; Ma, Wei Ji (2017) Bayesian microsaccade detection. J Vis 17:13
Shin, Hongsup; Zou, Qijia; Ma, Wei Ji (2017) The effects of delay duration on visual working memory for orientation. J Vis 17:10
Bhardwaj, Manisha; van den Berg, Ronald; Ma, Wei Ji et al. (2016) Do People Take Stimulus Correlations into Account in Visual Search? PLoS One 11:e0149402
Shin, Hongsup; Ma, Wei Ji (2016) Crowdsourced single-trial probes of visual working memory for irrelevant features. J Vis 16:10
Shen, Shan; Ma, Wei Ji (2016) A detailed comparison of optimality and simplicity in perceptual decision making. Psychol Rev 123:452-80

Showing the most recent 10 out of 29 publications