The research objective of this award is to develop and evaluate methods to create CAD geometry using natural and fluid means of human-computer interaction. The first goal is to en-able engineers to describe the structure of a conceptual design by sketching, gesturing, and talk-ing, in much the same way they ordinarily communicate with each other. The second goal is to take the multimodal description (sketch, gesture, utterance) of the desired behavior, and have the system infer the geometric constraints necessary to produce that behavior. The approach to un-derstanding multimodal design descriptions will rely on a blackboard architecture and mutual disambiguation. Each of the modalities will contribute hypotheses about the meaning of the cur-rent utterance/gesture/sketch, and cross-modal inference will prune the possible interpretations. This will provide a robust, flexible, general-purpose foundation for merging modalities. The techniques for transforming descriptions of behavior and rough structure into working geometry will make it possible to derive constraints on the geometry of a device sufficient to ensure a de-sired kinematic behavior. Combining these techniques with a design-by-features approach will enable generation of 3D geometric models from sketches. Here both the modeling process and the output are novel: the process is one of natural description of structure and behavior, and the output includes behavior-ensuring constraints that are automatically inferred from that descrip-tion.
If successful, this research will result in techniques which go beyond the limits imposed by traditional mouse-based interfaces to allow designers to create functioning geometry from in-formal device descriptions. These techniques will enable designers to focus more directly on be-havior, rather than structure, resulting in unique and novel product designs. Results from this re-search will be widely disseminated, demonstrated to students in grades 4-8 at Science and Tech-nology Education Partnership annual conferences, and incorporated into undergraduate engineer-ing courses.
