We have made progress on four projects: 1. The organization and operation of inferior temporal cortex. This project builds on two projects in the lab. The first examined the extent to which neurons in posterior inferior temporal cortex (IT) represent perceptual color space (Bohon et al 2016, eNeuro). The second, measured fMRI responses to a large battery of visual stimuli and uncovered a fundamental organizational plan of IT, which can be characterized as multi-stage parallel processing networks (Lafer-Sousa et al, Nature Neuroscience, 2013; Verhoef et al, Journal of Neuroscience, 2015; Lafer-Sousa et al, Journal of Neuroscience, 2016; Conway, Annual Review of Vision Science, 2018). The present question concerns the role of IT in object recognition, and the extent to which color supports those computations. We hypothesized that objects show distinct color properties, and that IT uses this information towards object recognition. To test this hypothesis, we examined the color statistics of objects in a database of 20,000 images identified independently by human observers from among 200,000 images. We show that objects, either naturally colored or artificially colored, could be classified from backgrounds using only the average color across the object. Objects tended to be warmer colored (L>M) and more saturated than backgrounds, which were cooler colored (M>L) and desaturated. That the distinguishing chromatic properties of objects rely on the L-M post-receptoral mechanism, rather than the S mechanism, is consistent with the idea that trichromatic color vision evolved in response to a selective pressure to identify objects. We also found that classifiers trained using only color information could distinguish animate versus inanimate categories; this classification depended predominantly on color modulation along the S axis. These results suggest that color could contribute to the IT computation of animacy, a computation that has previously been thought to depend solely on object shape. Object-color probabilities were correlated with the firing rate of neurons recorded in fMRI-identified color-biased regions of IT, and with fMRI signals recorded across IT, but not in V1. These results were published in the Journal of Vision (Rosenthal et al., 2018), and relate to our prior work done in collaboration with Ted Gibson, in which we showed that across languages, warm colors are communicated more effectively than cool colors (Gibson et al, PNAS, 2017). 2. Stimulus-driven functional organization of frontal cortex in macaque monkey Motivated by our work showing a global organizational plan of IT comprising parallel, multi-stage processing streams for faces, colors, objects and scenes, we sought to investigate the stimulus-driven organization of prefrontal cortex (PFC), one of the main output targets of IT. We measured fMRI responses to color, scene, disparity, face, and eccentricity stimuli in the same individual animal subjects, providing quantitative evidence for stimulus-driven color-biased regions in macaque PFC and knowledge of the relative locations of different functional domains, controlling for individual differences in absolute location within the PFC. Two color-biased regions were observed: an anterior patch located in anatomically defined 47/12 and a posterior patch in 8Av. These regions were adjacent to and non-overlapping with face-responsive patches; they showed lower selectivity for color than face patches showed for faces. Both sets of regions showed lower selectivity than the corresponding regions of IT measured in the same animals. We also found two scene-biased regions, also in 47/12 and 8Av, and a disparity-biased region in 8Av. In most animals, the regions defined by a bias for color, scenes or disparity were non-overlapping with the face-biased regions, but were at least partially overlapping with each other and showed a peripheral visual field bias. Face patches showed little response to the eccentricity stimuli. The functional domains were not neatly defined by anatomical partitions of frontal cortex. These results suggest that PFC comprises a functional organization that is independent of task demands. This work was published in Neuroimage (Haile et al., 2019). 3. Comparing cortical organization for pitch in monkeys and humans: the organization of the visual system is almost identical in monkeys and humans; is this true for the auditory system? Pitch is a critical component of human hearing that preferentially engages stereotyped regions of human auditory cortex. Because pitch is prominent in vocal sounds, the mechanisms of pitch perception are often presumed to be conserved across species. But neural responses to pitch have not been assessed with the same methods in human and non-human animals, so it remains unclear whether different species use similar mechanisms. Here we used fMRI to directly compare cortical responses to pitch in humans and macaque monkeys. Responses to pitch were weak or absent in macaques, even when tested with ecologically relevant stimuli (macaque vocalizations). Weak pitch responses could not be explained by poor data quality, because robust frequency-selective responses, organized tonotopically, were observed in the same data. These findings provide a rare example of a substantial difference in the functional organization of sensory cortex between two closely related primate species. This work was published in Nature Neuroscience (Norman-Haignere et al, 2019). 4. A paradoxical memory color for faces: the importance of color for social cognition. Color provides information about many kinds of objects including faces, but the specific benefit of color has been surprisingly difficult to pin down many influential models of vision simply ignore color. The predominant view is that color evolved for detecting fruit. Alternatively, color may have evolved for social communication, to transmit information about emotion, health, and sex. Here we address the relative contribution of color to object and face perception by testing memory colors. Participants were asked to match the color of real objects under two conditions: normal white light, and monochromatic sodium light that renders vision objectively achromatic. Under white light, color judgements of faces were more accurate than color judgements of fruit and other familiar objects. Under sodium light, judgments of all objects were mostly achromatic except faces, which appeared strikingly green. This work was published in Nature Communications (Hasantash et al., 2019).
| Conway, Bevil R; Eskew Jr, Rhea T; Martin, Paul R et al. (2018) A tour of contemporary color vision research. Vision Res : |
| Conway, Bevil R (2018) The Organization and Operation of Inferior Temporal Cortex. Annu Rev Vis Sci 4:381-402 |
| Lafer-Sousa, Rosa; Conway, Bevil R (2017) #TheDress: Categorical perception of an ambiguous color image. J Vis 17:25 |
| Gibson, Edward; Futrell, Richard; Jara-Ettinger, Julian et al. (2017) Color naming across languages reflects color use. Proc Natl Acad Sci U S A 114:10785-10790 |
