In contrast to the prevalent uni-modal perspective, the project focuses on quantitatively integrating information residing in multiple modalities to yield observational behavior analysis descriptions that range from global categories to time continuous behavioral abstractions and contributes novel algorithms and models for recognizing and modeling communicative and affective interaction dynamics elicited in realistic settings of couples and family therapy. The computational challenges are multiple from automated perception of emotionally rich behaviors and cognition through models for domain specific interpretation of the sensed information, to action through combining the knowledge and expertise of humans with the information processing abilities of the machine.
The research impacts a wide range of applications centered on observations of the human state and interactions, e.g., mental health applications, business customer services, negotiation tactics, law enforcement (interviews), etc. The project also provides multi-disciplinary training for undergraduate and graduate students. The expected outcomes include benefits to psychology through novel information augmentation, in technology through improved intelligent and robust human behavior computing, and in observational practice through the introduction of transformational tools.
