Communication is a dynamic process by which information is transferred across people. For the sake of experimental control, however, most cognitive neuroscience work on communication focus on either language production in the speaker's brain or comprehension in the listener's brain during highly artificial tasks. Communication, which by nature is a joint action embedded in a social context, is paradoxically studied in single individuals in isolation. In this proposal, we advance a new and versatile framework for understanding the neural mechanisms underlying communication in the real world. This framework argues that effective communication emerges when the neural activity of the two interlocutors are ?coupled? together. This coupling can take the form of (1) mirroring, when the listener's neural patterns match those of the speaker; (2) conditional transformations, when the listener's patterns reflect a lawful relation to the speaker's neural patterns; or (3) synergies, when the activities of the two brains dynamically influence and constrain each other to optimize information sharing. To test our theoretical framework, we propose developing both stationary and portable dual-brain imaging systems for measuring the neural activity of multiple individuals engaged in dialogue in the laboratory and in clinical settings. Two of the systems, fMRI hyperscanning and ECoG hyperscanning, take advantage of the high spatial and high temporal resolution of the respective methods to precisely characterize coupled neural dynamics during dialogue. For the third system, we propose developing a portable, dual-brain fNIRS system to characterize how two brains interact in real-life settings. Field work measuring the level of brain-to-brain coupling between a caregiver and a child could be used to study the acquisition of a first language; as an early preverbal biomarker for developmental disorder (e.g. lack of caregiver-child coupling as an early biomarker for autism); and as a temporally refined diagnostic tool for evaluating the effectiveness of behavioral and pharmacological interventions aimed at alleviating communication deficits (e.g. in autism, schizophrenia). Although the proposed research is technologically challenging, this laboratory has a track record of developing innovative analysis tools and theoretical frameworks for the study of cognitive functions in real-life contexts. The PI has substantial experience working with ECoG, fMRI and fNIRS, as well as studying both neurotypical and clinical populations. Our proposal is strongly grounded in prior work studying the extent of shared neural responses across subjects during the processing of real-life information. Thus, although ambitious, the research plan is both feasible and grounded, and has the potential to transform the way we understand and assess the neural processes by which we interact with others in everyday contexts. Ultimately, we believe that this work will lead to a novel brain-to-brain coupling biomarker for detecting preverbal communication disorders and assessing interventions in clinical settings.
We propose that communication leads to the coupling of neural responses in speakers and listeners, and that such neural coupling is necessary for successful communication. This ?brain-to-brain coupling? will be investigated during natural conversations in real-life situations (e.g. caregiver-child, clinician-autistic, therapist- patient, conversations) using stationary and portable dual-brain imaging systems that simultaneously measures neural activity in both participants. Such measurements can provide a valuable biomarker for understanding and characterizing miscommunication, communication disorders, and developmental disorders, as well as a tool for assessing the effectiveness of psychological and pharmacological interventions designed to improve communication.
| Haufe, Stefan; DeGuzman, Paul; Henin, Simon et al. (2018) Elucidating relations between fMRI, ECoG, and EEG through a common natural stimulus. Neuroimage 179:79-91 |
| Liu, Yichuan; Piazza, Elise A; Simony, Erez et al. (2017) Measuring speaker-listener neural coupling with functional near infrared spectroscopy. Sci Rep 7:43293 |
