This is a three-year continuing award. Collaboration is a hallmark of human activity. But today's society is becoming global, and digital networking-both wired and wireless-aims to serve geographically-dispersed participants. Networks of computers now become a mediating tool in human collaboration. And, their distributed architecture provides new opportunities for expanding human intellect and for enhancing knowledge work. But to be maximally useful the mediating system must be easy to use-preferably transparent to the user. Communication between human and machine should approach the naturalness of face-to-face exchange, where the sensory modalities of sight, sound and touch carry primary responsibilities for information transport. Additionally, supporting software must include intelligent agents able to fuse sensory data into meaningful commands for the machines. New technologies, now evolving, promise greater naturalness in human/machine communication-naturalness that substantially transcends the limitations of traditional mouse and keyboard. Incorporated with audio and video conferencing, these techniques include: eye tracking, visual gesture, hands-free speech recognition, speech synthesis, force-feedback tactile grasp, and manual gesture. Used in natural combination, these modalities provide the human greater dimensions of control and communication, and hence greater effectiveness in networked collaboration.
This new research, aims to establish the basic understanding necessary for the creation and use of multimodal computer interfaces in wired and wireless networks. The research aims to produce a design methodology for multi-user collaboration across heterogeneous architectures and computing environments. As multimodal capabilities evolve, human performance must be addressed as a recognized component of the total system. Optimization of design therefore includes human-factors assessment to quantify the synergies that can be won from multimodal interaction. The new research identifies three application sectors where wired/wireless multimodal collaboration can provide new solutions; they include: Telemedicine (where expert participants may contribute to diagnoses for remote patients); Crisis Management (where disaster relief and emergency aid may be planned and deployed collaboratively); and Mobile Offices (where business activities might be conducted in transit-air, sea or land). The design framework resulting from this research is also applicable to other activities important in today's society, including: ubiquitous and universal access to information systems; increased interaction with the World Wide Web; education and knowledge dissemination; and, new aids for disabled individuals.
