This project proposes novel strategies for the automatic capture of teamwork in crowded spaces. The initial goal is to develop new methods for acquisition of people and artifact locations using diverse modalities, including machine vision and radiofrequency identification (RFID). Following, software will be developed in order to integrate these observations and track team members and artifacts over time. Ultimately probabilistic reasoning will be used to identify team tasks based on these unified observations. The representative domain of trauma resuscitation is an ideal environment for this work since the roles and tasks of players are well-defined and the flow of work follows a general schema regardless of the patient's injuries. The system will be tested in simulated trauma scenarios using a robotic mannequin patient.
There are two key benefits of this work. First, the process of deriving system requirements for computerized teamwork support systems demands analyzing a large number of observations of current practices. Automatic transcription and tagging of teamwork will allow for efficient capture and interpretation of events and is preferable to more tedious and error-prone observations by experts. Second, automatic tracking of team activities is a needed initial step in the development of "smart rooms" that provide computerized support of teamwork.
The core contribution of this project will be a proof-of-concept system integrating tracking of actors and medical objects using computer vision and RFID tracking. The proposed approach will develop novel algorithms and methods for (i) vision-based person tracking in crowded collaborative environments and (ii) fusion of unreliable data from multimodal sensors to achieve reliable recognition of human activities.
Broader Impacts The scientific importance of this work is in the need to tag video observations. Many forms of videos are of repetitive behaviors, whether in surveillance applications, work situations, or other uses. In all such cases, applying a grammar to the video, and matching actions and sounds to that grammar, has the possibility of greatly simplifying the job of work analysis - a critical phase in the development process of computer support for complex, high-risk human activities. This work will also provide the foundation for implementing decision aids in environments such as trauma resuscitation and related medical domains that lack effective methods for instrument tracking of work.
