The goal of this Early-concept Grant for Exploratory Research is to explore a new generation of computational tools for joint modeling of physiological and linguistic signals of human behavior. The project is the first to investigate physio-linguistic models for deception analysis. To achieve this goal, the following three research objectives are pursued. First, a novel physio-linguistic dataset of deceit is built, covering several different domains. Second, rule-based classifiers for deception detection are explored, using physiological features (e.g., heart rate, respiration rate, galvanic skin response, skin temperature), as well as linguistic features. Third, data-driven learning approaches for multimodal deception detection are developed, taking advantage of the recent progress in early, late, and temporal fusion models.
The project is exploratory in nature, and acts as a catalyst for novel research problems. First, it explores rich sets of multimodal features extracted from physiological and linguistic modalities, analyzing their effectiveness in the recognition of deceit. Second, it also explores the integration of multiple physio-linguistic modalities, through experiments with rule-based and data-driven techniques that fuse multimodal features into joint deception analysis models. To address the challenges of multimodal research work, the team working on this project brings together experts from the fields of bio-sensors, computational linguistics, and physiology and behavioral sciences.
The project has high potential payoffs, as models of deception detection have broad applicability, including: the development of critical tools for various applications in fields such as criminal justice, intelligence, and security; the enhancement of applications that can be negatively affected by the presence of deceit, such as opinion analysis or modeling of human communication; and a deeper understanding of fundamental aspects of human behavior, which can positively impact medical applications in psychiatry and psychology. The tools and datasets produced during this project will be made freely available for the research community.
For further information see the project web site at: http://web.eecs.umich.edu/~mihalcea/deceptiondetection/
