This project will formulate a set of guidelines for conducting empirical studies to evaluate formal methods for idea generation in collaborative environments. In order to conduct good experiments on idea generation methods, a clear understanding of experiment variables, method variables, and data collection variables is needed. Also, a framework to analyze the data collected from such experiments is required. The research is divided into four major tasks: survey and evaluation of creative cognition models; identification of operation variables, metrics, and experiment design; development of a framework for analysis of design sketches; analysis and critique of previous experiments on progressive idea generation (C-Sketch, Method 635, and Gallery Method). This project will investigate the issue of scaling up appropriate cognitive models to study typical design problems which are far more complex than those used in psychological experiments. This study will attempt to understand the correlation between four sets of factors relevant to collaborative design: the individuals, the situation/environment, the outcome, and the process. This research will also develop a methodology for analyzing design sketches for studying the cognitive processes, mental structures, and ideas generated by tracking the evolution of design concept at successive stages of development. Studies have shown that the early stages of conceptual design have a substantial impact on the efficacy of a product; development of effective design methods to support the generation of alternative design ideas is critical. Along with that, designers must be convinced that the use of these methods has potential benefits. Therefore, it is essential to collect and analyze data on the effectiveness of idea generation techniques. The availability of experiment methodology and metrics for evaluating and/or optimizing creative design methods is an important step in this direction. If successful, future research will employ these guidelines to produce verifiable database of performance of formal creativity methods which may eventually lead to wider use of effective idea generation methods to explore alternative designs.
