Gestures, deictic referencing (pointing at objects and regions of interest), and manipulation of digital images are an integral aspect of decision making in collaborative scientific and medical work. In modern minimally invasive surgical interventions, medical imaging has come to play an increasingly important role, but due to concerns of asepsis image manipulation during surgical decision making are typically constrained. In this project, the PI will explore the use of new technology such as the Kinect to address this issue, by developing techniques for "touchless" interaction to coordinate and enhance communication among team members in the operating room. The results of this research will provide a deeper understanding of collaborative practices around image use and the benefit of technological tools for annotating and referencing those images, which will significantly benefit patient outcomes. The findings will have broad impact, in that they will be translatable to other expert collaborative environments that utilize imagery in order to have a positive effect on team work practices. The PI will disseminate project outcomes broadly to both the medical and the human-computer interaction (HCI) communities.
An understanding of how new imaging interaction techniques can be integrated into and, in turn, shape coordinated practice will constitute an important contribution to the state of the art. To achieve these goals, the PI will identify coordinated practices and their relationship to imaging use through a detailed field study of laparoscopic surgery. She will then iteratively design and implement a gestural image annotation prototype for laparoscopic surgeons to reference and annotate endoscopic video. And she will determine the effects of the imaging manipulation on coordinated surgical practice through an experimental study.
