To produce high quality solutions during complex problem solving, groups must be able to realize effective problem solving processes including the ability to quickly create, share, and exchange task artifacts, the ability to maintain awareness of each other's activity, and the ability to transition between performing individual work in parallel and joint work in a shared space. Yet despite many years of research, groups are still unable to realize effective processes when using digital artifacts and their supporting devices. This can severely inhibit a group's ability to produce quality solutions. Though not new, this problem is becoming extremely urgent as more users are collaborating with digital artifacts to solve increasingly complex problems in critical domains such as software, design, and security. In this project, the PI will investigate one promising approach for facilitating group problem solving: multi-user, multi-display environments (MDEs). An MDE networks personal and shared devices to form a single virtual workspace. Personal devices such as laptops provide physically separate spaces for performing individual work in parallel, while large displays provide a shared visual space for joint work. But these spaces must be integrated through enabling systems and supporting interactions. While many enabling systems exist, designing interfaces, interactions, and visualizations (collectively comprising the interaction framework), which combine to facilitate effective group problem solving in an MDE, remains a grand challenge. In this project, building upon the current state-of-the-art in our theoretical understanding of group work and his extensive multidisciplinary research experience, the PI will develop an MDE interaction framework consisting of three core components: a management interface, overview visualization, and input redirection. The management interface will enable a user to relocate applications among displays in the MDE while minimizing disruption to other ongoing work, yet also preventing when desired others from relocating or interacting with given applications on a shared display. The overview visualization will enable users to maintain awareness of each other's ongoing work relative to the central activity. Input redirection to any shared display will allow users to jointly interact with applications on those displays. The PI will study how these core components interrelate and affect in combination the ability of an MDE to support effective group problem solving. A successful outcome will facilitate effective group problem solving by allowing users to create, share, and exchange task artifacts, to maintain awareness of each other's activity, and to seamlessly transition between individual and joint work.
Broader Impact: The PI will disseminate the software developed as part of the research, to enable end users to utilize MDEs for their own problem solving activities and researchers to further investigate techniques for facilitating effective group work in MDEs. The empirical results and lessons learned from the project will advance scientific understanding of how to develop interaction frameworks for MDEs that allow groups to better realize effective problem solving processes. By enabling more effective processes, the longer-term impact of the work is that groups will be able to create higher quality solutions for complex problems more of the time. The various research activities will be tightly integrated by the PI into his courses at both the graduate and undergraduate level.
A multi-user, multi-display environment (called an MDE) represents a transformative approach for enabling face-to-face group problem solving in business and academic settings. An MDE networks personal and shared devices to form a virtual workspace. Personal devices such as laptops and tablets provide separate spaces for performing individual work while large displays provide a shared visual space for joint work. The formation and use of MDEs is increasingly possible due to the proliferation of small and large devices and wireless networks. For example, most conference rooms and other types of meeting spaces now have large projected displays and people often come into the meeting spaces with their own laptops. However, effective use of MDEs presents many fundamental challenges for human-computer interaction. For instance, how does a user move a Web browser from his or her laptop to a shared display? And how can other users be allowed to jointly interact with the browser while it is on the shared display? The purpose of this project was to investigate these and many other challenges of human-computer interaction in MDEs. The intellectual merit of this project produced the following scientific outcomes: Empirically-based lessons for rapidly prototyping new interactions and user interfaces in MDEs. The application of these lessons within existing design processes can yield more effective and efficient testing of preliminary designs and prototypes. The invention of a new interaction metaphor called a world-in-miniature interface for visually manipulating applications and mouse / keyboard input within an MDE. For example, the interface enables a user to move an application window between two distant displays without having to leave the local device. The key innovation is to visualize the workspace using a top-down, fold-out view of the meeting space. Empirical results and lessons supporting the particular design representation of the world-in-miniature metaphor. The invention of new interaction techniques that promote better awareness and scalability of the world-in-miniature metaphor. For instance, the new techniques allow users to have many applications running in the MDE without sacrificing the ability to manipulate or control any one individual application. Empirical results and lessons showing the superior effectiveness of the world-in-miniature metaphor relative to commonly used alternatives for MDEs, including a text-based interface and an interactive, top-down view (without a fold-out view) of the workspace. A comprehensive revision to the world-in-miniature metaphor for use within MDEs based on several years of continued research, use, and testing. A key innovation of the revision was to better integrate the world-in-miniature metaphor with the traditional desktop metaphor. Results and lessons from the first scientific study of the use of an MDE in a real-world setting. The task setting was collaborative face-to-face software engineering within a large software company based in the U.S. The broader impacts of this project include: The completion of one doctoral dissertation in Computer Science The completion of five research-oriented projects for M.S. degrees in Computer Science Many project opportunities for undergraduate research in Computer Science The integration of case examples and lessons from the research into an undergraduate course on user interface design. Influencing the design of user interfaces for sharing information on interactive tabletops. Many of the early interfaces for collaborative use of tabletops adapted the concept of the world-in-miniature interface invented in this research.
