The complexity of natural images is potentially enormous: the number of possible images that can be described by a smallish (100 by 100 pixels) picture is practically infinite (10000256), more than all the images the human race has ever witnessed during its entire existence. How can any system process input data of this magnitude of dimensions and interpret/understand it in terms of the estimated 200,000 objects in the world, their spatial layouts, and scene structures? Yet, this is a task that human visual systems routinely perform in a fraction of a second. The secret must lie in the fact that natural images are highly redundant, living in a restricted space inside this universe of almost infinite possibilities, and that mammalian visual systems have discovered and exploited this fact. In particular, we conjecture that neurons and populations are tuned to the statistical structure of natural images, building on previous work showing, for example, that sparse coding ideas can help predict receptive field properties of 'simple cells'in the visual cortex. This proposal has three stages. Firstly, we will perform a statistical analysis of natural images to classify and model the types of visual patches that appear. This will result in a stimulus dictionary, which will be used as stimuli to investigate the tuning properties of neurons and neuronal populations, and a visual concept dictionary which will be used to make predictions for the tuning properties. Secondly, we will perform multielectrode neurophysiological investigation of the tuning properties of neurons, and neuron populations, at different levels of the visual cortex in response to the stimulus dictionary. Thirdly, we will perform data analysis to model the tuning properties of neurons and populations using a combination of model-driven, which assumes that neurons are tuned to statistical properties of images, and data-driven approaches which can be thought of as learning 'neural visual concepts'directly from the neuron's response to the stimuli. Our theoretical approach - for learning the stimuli dictionary, the visual concepts, and performing data analysis - is based on statistical and machine learning techniques. These assume a hierarchical compositional structure for the data which offers the possibility of taming the complexity of natural images and is also consistent with the known hierarchical structure of the visual cortex. Intellectual merit: This research will help understand the structure of natural images, determine models for the tuning properties of neurons in the visual cortex, and develop novel data analysis techniques. It has the potential to significantly advance our understanding of the statistical structures of natural images and the neural encoding of these structures, including the population level. This will lead to greater understanding of the visual cortex and also help the development of computer vision systems. Broader impacts: This project is interdisciplinary in nature and should have broad impact in multiple disciplines: neuroscience and biological vision, statistical neural data analysis, computer vision, and machine learning. Understanding neuronal properties in the visual cortex is a pre-requisite to the clinical enterprise of developing therapeutic methods and prosthetic devices for the visually impaired. The proposed research program will help facilitate a new graduate program in Computational and Cognitive Neuroscience at UCLA, an inter-college undergraduate minor in Neural Computation at CMU that the investigators are developing at their respective universities. The investigators also plan to organize workshops in NIPS, COSYNE, as well as to integrate their research into both undergraduate and graduate curriculum in their respective universities. This work will also affect undergraduate students at other colleges, by a summer undergraduate training program in Pittsburgh, another at CMU's Qatar campus. In addition, we will propose a workshop and summer school at IPAM (UCLA). We anticipate that this research will lead to invited lectures, peer reviewed publications and, if successful, will have national and international impact. The PIs have good track record in involving undergraduates, including women and minorities, in their NSF-sponsored research, and will continue to endeavor in the training of the next generation of computational neuroscientists.
This project will lead to greater understanding of neural mechanisms and coding strategies in the primate visual cortex. Such knowledge is fundamental to understanding human visual functions and is critical to the clinical enterprise of developing better diagnostic tools, therapeutic methods, and prosthetic devices for the visually impaired.
| Samonds, Jason M; Tyler, Christopher W; Lee, Tai Sing (2017) Evidence of Stereoscopic Surface Disambiguation in the Responses of V1 Neurons. Cereb Cortex 27:2260-2275 |
| Zhang, Yimeng; Li, Xiong; Samonds, Jason M et al. (2016) Relating functional connectivity in V1 neural circuits and 3D natural scenes using Boltzmann machines. Vision Res 120:121-31 |
| Ma, Jiayi; Zhao, Ji; Yuille, Alan L (2016) Non-Rigid Point Set Registration by Preserving Global and Local Structures. IEEE Trans Image Process 25:53-64 |
| Lee, Tai Sing (2015) The visual system's internal model of the world. Proc IEEE Inst Electr Electron Eng 103:1359-1378 |
| Mao, Junhua; Zhu, Jun; Yuille, Alan L (2014) An Active Patch Model for Real World Texture and Appearance Classification. Comput Vis ECCV 8691:140-155 |
| Zhu, Yu; Zhang, Yanning; Yuille, Alan L (2014) Single Image Super-resolution using Deformable Patches. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2014:2917-2924 |
| Papandreou, George; Chen, Liang-Chieh; Yuille, Alan L (2014) Modeling Image Patches with a Generic Dictionary of Mini-Epitomes. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2014:2059-2066 |
| Samonds, Jason M; Potetz, Brian R; Lee, Tai Sing (2014) Sample skewness as a statistical measurement of neuronal tuning sharpness. Neural Comput 26:860-906 |
| Jiayi Ma; Ji Zhao; Jinwen Tian et al. (2014) Robust point matching via vector field consensus. IEEE Trans Image Process 23:1706-21 |
| Chen, Liang-Chieh; Papandreou, George; Yuille, Alan L (2013) Learning a Dictionary of Shape Epitomes with Applications to Image Labeling. Proc IEEE Int Conf Comput Vis 2013:337-344 |
Showing the most recent 10 out of 13 publications
