This research develops new tools that redefine the roles of content and technology to promote discovery in learning. Information discovery begins with finding information resources that are relevant to a problem; it continues through recognizing connections between ideas that the resources represent, and synthesizing new ideas. While current search engines help people find relevant information, they present results as fixed lists of text, which are weak cognitive surrogates for the information that they represent. Representations that combine images and text function as better stimuli for understanding, navigation, learning, and discovery. Further, manipulation of representations helps people to see relationships between elements, and thus enables the cognitive restructuring that is an essential stage of discovery. Composition is the process of putting elements together in new structures, with intentional relationships, to form a whole. To promote information discovery, new tools must connect searching and browsing with collecting and composing, and thus fluidly integrate processes of information finding, understanding, and restructuring.
This project will develop and study a mixed-initiative system using integrated image-text surrogates, aimed at promoting information discovery in learning. In mixed-initiative systems, agents work concurrently and proactively with the user to accomplish a task. This experimental system will form image-text surrogates to represent documents using a semantic model, and select those that are most interesting to the user. The system will be deployed in a course on design and invention ("The Design Process") taken by science, engineering, and business students, where its impact on discovery and learning will be assessed.
This interdisciplinary research connects methods from education, library science, cognitive psychology (including creative cognition), human computer interaction, digital libraries, visual art and design, information retrieval, and pattern recognition. Expected outcomes include (1) integration of information retrieval and pattern recognition methods with novel advanced visual interfaces; (2) new automatic methods for representing documents with integrated image-text surrogates; (3) new insights about the practices and needs of inventors and other design innovators; and (4) new methods for integrating creativity into education. The advanced leaning technologies will be implemented and distributed through the combinFormation software platform for mixed-initiative browsing, searching, collection, and composition, which will be used annually on assignments by 1000 students in The Design Process course, and made freely available to the public through the internet
The award enabled development of new methods for supporting creativity in people working with information. Information discovery tasks are human endeavors in which the goal is to develop new ideas. Examples span conceptualizing an approach to an interdisciplinary topic like sustainability, planning an evening, vacation, summer internship, or career, and initiating a thesis or invention. Information composition is a holistic representation for collections, integrating visual and semantic representations to help people see, understand, and develop connections amidst an information collection. In mixed-initiative information composition, computational agents proactively work for the human. The user specifies topics to research. The agents must find relevant documents, extract relevant clippings, and visualize these in the information composition. We call the clippings surrogates, because each functions as a visual bookmark, enabling navigation back to its source document. The extraction of relevant image and text surrogates is a key stage of mixed-initiative information composition, because these are the representations that a person sees and manipulates. It posed a challenging research problem. The research developed new methods for recognizing informative parts of web pages, and extracting informative image and text surrogates from them. The methods are algorithms, that is, step-by-step procedures a computer executes to accomplish a task. A passage of text is associated with each image, providing context for visualization algorithms that organize the incoming stream of surrogates. The image-text surrogate recognition algorithm was incorporated into the combinFormation mixed-initiative information composition software [http://ecologylab.net/cf]. There, the surrogates flow into the information composition visualization, where a human can select, organize and design those relevant to the information discovery task at hand. combinFormation was used by hundreds of students on assignments each semester, for developing prior work collections for projects and inventions. We built on creative cognition research to develop new metrics for assessing the creativity of information compositions. These metrics were used in the courses, and in laboratory experiments.
