Working memory is a human cognitive mechanism that is involved in solving problems and making decisions, which is known to have a limited capacity of 7 +/- 2 units, where the complexity of each unit can vary from a single binary digit to a chunk that corresponds to a mental model. It has further been shown that people form better mental models in response to labeled images than they do in response to textual elements without accompanying images. The information age is the age of collecting. People collect web page bookmarks, articles, email, audio, and video. Digital libraries collect texts and multimedia. Search engines create collections dynamically. Collections are comprised of sets of records, and are typically presented in the form of lists of hyperlinks, in most cases lists of textual elements. Information dependent tasks, such as buying a car or writing a research paper require users to make conceptual connections between elements. We need to discover how to represent collections of information elements to account for the limitations of working memory, promote the formation of mental models, and support user tasks. Composition is a visual technique for forming chunks of conceptually related elements. In a navigational hypermedia composition, each visual information element functions as a navigational link to an information resource. The PIs' hypothesis is that presenting bookmark and result sets as compositions of labeled image elements will mitigate the demands placed upon the user's working memory. The PIs' long-term goal is to develop an understanding of the cognitive mechanisms of information seeking, so that they will be able to build tools that enable users to develop collections in forms that support the cognitive structures and processes involved in knowledge building. In this project, the PIs will gather preliminary data to establish that working memory function is extended by presenting bookmark and result set collections as visual compositions. Specifically, they will: evaluate bookmark and result set representations with divergent as well as convergent thinking tasks; support the formation of mental models by presenting collections with labeled images; and extend working memory capacity by presenting collections as compositions of visually blended labeled images. To these ends, they will take advantage of findings from cognitive science, and techniques from information retrieval, computer graphics, visual art, and design.
Broader Impacts: Current approaches to presenting bookmark and result sets as textual lists do not optimize our ability to form mental models, and maintain them in working memory. Information workers and students throughout academia, government, and industry often need to connect multiple idea sources, in the course of knowledge building activities such as problem solving, discovery, and writing. Because information workers spend so much time dealing with bookmark and result sets, the result is a waste of intellectual capacity across the knowledge economy. By presenting collections in forms that optimize use of mental model and working memory capabilities, the PIs expect to improve productivity and spur innovation for a broad population of internet users.
