The enormous potential of sketch-based interfaces for conceptual engineering design is widely recognized and has been broadly pursued, but the practical application of these systems has remained limited because they are two-dimensional and do not embody functionality. The PI has previously demonstrated algorithms based on statistical machine learning that allow reconstruction of a 3D object from a 2D freehand drawing directly. By extending and combining this work with recent mass-market pen-based hardware, the PI will in this project create the basis of new conceptual design tools for rapid visualization, simulation and learning for geometry-intensive domains. A new form of freehand sketching will be explored, in which the sketch itself is spatially reconstructed in interactive time and then can be physically simulated, a process the PI calls "live sketching." The PI's system will enable users to construct complex 3D models quickly and intuitively, to analyze them, to change viewpoints while sketching, and to communicate and collaborate on spatial concepts with other designers more easily. To this end, the PI will first extend his reconstruction algorithms through context sensitivity and temporal scene segmentation so they can handle more complex geometries than is currently possible, including curved surfaces. This effort will then be combined with existing simulation and analysis codes of various disciplines and will be integrated into a prototypical platform allowing construction of 3D models by successive piece-wise sketching, reconstruction, and interactive simulation. The resulting tools will fundamentally alter the way engineers interact with design systems, bringing together traditional computer-aided engineering and simulation tools so they can play a role much earlier in the critical preliminary design stages.
Broader Impacts: The outcomes of this project will positively impact any design domain where it is necessary to predict the behavior of spatial geometries, including many application areas of mechanical, civil, MEMS, and architectural CAD. Live sketching will also make engineering tools for modeling and simulation much more accessible to non-engineering designers, students and teachers, by removing interface barriers. The PI will in addition develop and make available online a live sketch application for teaching structure and kinematics, which will allow students and teachers to explore and demonstrate kinematics and structural analysis of trusses and solid structures by sketching structures, applying loads, and seeing the resulting mechanism trajectories, exaggerated deformations, and colored stress concentration for various shapes, in real time while sketching. This project will be made available on-line in open-source format, so that additional types of simulation can be integrated into it by others as required.
