To understand and navigate the environment, sensory systems must solve simultaneously two competing and challenging tasks: the segmentation of a sensory scene into individual objects and the grouping of elementary sensory features to build these objects. Understanding perceptual grouping and segmentation is therefore a major goal of sensory neuroscience, and it is central to advancing artificial perceptual systems that can help restore impaired vision. To make progress in understanding image segmentation and improving algorithms, this project combines two key components. First, a new experimental paradigm that allows for well-controlled measurements of perceptual segmentation of natural images. This addresses a major limitation of existing data that are either restricted to artificial stimuli, or, for natural images, rely on manual labeling and conflate perceptual, motor, and cognitive factors. Second, this project involves developing and testing a computational framework that accommodates bottom-up information about image statistics and top-down information about objects and behavioral goals. This is in contrast with the paradigmatic view of visual processing as a feedforward cascade of feature detectors, that has long dominated computer vision algorithms and our understanding of visual processing. The proposed approach builds instead on the influential theory that perception requires probabilistic inference to extract meaning from ambiguous sensory inputs. Segmentation is a prime example of inference on ambiguous inputs: the pixels of an image often cannot be labeled with certainty as grouped or segmented. This project will test the hypothesis that human visual segmentation is a process of hierarchical probabilistic inference.
Specific Aim 1 will determine whether the measured variability of human segmentations reflects the uncertainty predicted by the model, as required for well-calibrated probabilistic inference.
Specific Aim 2 addresses how feedforward and feedback processing in human segmentation contribute to efficient integration of visual features across different levels of complexity, from small contours to object parts.
Specific Aim 3 will determine reciprocal interactions between perceptual segmentation and top-down influences including: semantic scene content; visual texture discrimination; and expectations reflecting environmental statistics. The proposed approach models these influences as Bayesian priors, and thus, if supported by the proposed experiments, will offer a unified framework to understand the integration of bottom-up and top- down influences in human segmentation of natural inputs.
This project aims to provide a unified understanding of perceptual segmentation and grouping of visual inputs encountered in the natural environment, through correct integration of the information contained in the visual inputs with top-down information about objects and behavioral goals. This understanding is central to advancing artificial perceptual systems that can help restore impaired vision in patient populations.
