The goal of this project is to develop an alternative technique for designing new objects using sketches and photographs; the particular application to be explored is architectural design. Rather than a "top down" approach, defining large masses in 3D and refining and decorating these masses, designs will be developed from sketches and photographs of varying levels of detail arranged by the user in a 3D environment. Unlike any current design system, the sketches and images do not need to represent geometrically- consistent representations of the structure during the design phase. The designer can experiment with different variations in form and detail in a view without struggling with a system that automatically changes other views. This freedom is made possible by representing the design as strokes and pixels only. The system will allow the user to tour the 2D sketches and photographs in a manner that lets the model emerge by the fusion of images by the user's own visual system. After the design is refined by multiple iterations of sketches that converge to a nearly consistent representation, computer vision techniques could be used to extract a 3D model from the users input. The advantage of the approach is that the user works in a familiar 2D mode for defining details, without the limitations and time delays of a full 3D model being defined at each step in the process. Intellectual Merit: In exploring the middle ground between sketch and object, the proposed work addresses one of the "holy grail" problems in interactive 3D graphics how to move seamlessly between 2D and 3D. This investigation will advance the understanding of how humans conceive of and design geometric form (including the abstraction of form and simplification of complex problems), an activity that shapes most of the objects with which we live and work. This understanding includes not only a computer graphics perspective (algorithms and representations) but also an architectural perspective, where practitioners have a deep working knowledge of form creation but lack a computational or mathematical background to rigorously analyze and describe it. Broader Impacts: The proposed work will have a direct impact on the creative design process by offering a novel approach for creating and editing 3D form, and will also impact the communication of ideas, where 2D representation has largely dominated. This work crosses and links disciplines, not only computer graphics and architecture, but other research areas related to how humans think (such as computer vision, cognitive science, psychology) and build (industrial design, civil engineering, mechanical engineering). The PI's close relationship with faculty, students, and practitioners at the Yale School of Architecture will provide opportunity to explore and test the new approach through application to real design problems.
