People can perceive the shape of objects accurately and reliably but how this occurs is not yet understood. This ability may stem, at least in part, from our use of both visual information and haptic information (information obtained when an object is touched or grasped). Moreover, if we learn to recognize an object based on visual information, we can often recognize the same object when our eyes are closed but we are allowed to grasp it. Similarly, if we learn to recognize an object based on haptic information, we can often recognize the object when we see it but cannot touch it. In other words, we exhibit cross-modal transfer of object shape information. How does information from the eyes and hands link up in the brain to yield a coherent representation of object shape? Insights obtained from this research can contribute both to our understanding of how humans perceive object shape using vision and/or touch and to development of improved robotic and other artificial intelligence systems operating in multi-modal settings in industrial, medical, military, and other applications.
The present project develops a theory of visual-haptic object shape perception in which people's notions of object similarity are not based on sensory features but rather on latent or hidden variables that represent object parts and their spatial relations in an abstract, modality-independent format. Object representations are formalized using a probabilistic "shape grammar" with Bayesian inference used to infer grammar-based object representations when an object is viewed, when it is grasped, or both. The model is tested using data obtained from behavioral studies of visual, haptic, and visual-haptic object shape perception by humans. The investigators will explore the types of representational change that underlie the transition from perceptual novice to expert (e.g, radiologists) and will assess whether perceptual expertise is well characterized as category learning, grammar learning, both, or neither. The research program will also develop a large public database of code for re-creating both visual and haptic features of complex objects. This will allow other researchers to fabricate the objects using a 3D printer, enhancing complementarity and comparison across research sites. Finally, training undergraduate and graduate students in the emerging field of computational cognitive science will contribute to a new generation of multidisciplinary scientists working across traditional boundaries between cognitive science and computer science.
