In order to interact appropriately with others, people need to recognize identities, facial expressions, and actions. What are the brain mechanisms underlying this recognition? This project proposes that face identity and expressions are processed by common brain regions, and that distinct regions are specialized to process static and dynamic information. This could be a particular case of a more general phenomenon: “complementarityâ€, in which different tasks like recognizing identity and recognizing expressions are performed by the same brain regions, because solving one of the tasks helps to solve the other. Complementarity might be a general principle of organization of the brain. This research aims to shed new light into social perception and more broadly into the large-scale organization of the human brain, through measurement of neural activity and building artificial neural networks. This aims to improve understanding of social cognition and its impairments, such as autism spectrum disorders. The project will design courses and research opportunities for students to receive training at the intersection between neuroscience and artificial intelligence (AI), preparing them to study neuroscience questions with new AI tools, and to build new AI inspired by state-of-the-art research in neuroscience. The project will also reach out to the broader community, using a combination of in-person and online events.
According to a classical theory, face identity and facial expressions are processed by separate neural pathways. The identity of a face is recognized by the ventral temporal lobe (occipital face area: OFA, fusiform face area: FFA), while expressions are recognized by the lateral temporal lobe (posterior superior temporal sulcus: pSTS). By contrast, recent research identified identity information in pSTS, and expression information in OFA and FFA. This project hypothesizes that: 1) representations of orthogonal properties like face identity and expressions operate synergistically during recognition, and arise spontaneously within the same brain regions; 2) this phenomenon occurs not only in the case of face identity and expressions, but also in the case of body identity and actions, constituting a broader principle of organization of social perception; 3) ventral and lateral regions are not distinguished by the content they encode, but by the type of properties they process. The project will test these hypotheses by asking participants to watch controlled and naturalistic videos of expressions and actions while undergoing fMRI, and by modeling their responses with encoding models using features from deep neural networks.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
