This research attempts to identify and measure the effect of perceptual and contextual features on sketch recognition, and use these results to create effective classifiers to recognize low level shapes and constraints that will identify all possible interpretations along with a ranking for use by higher-level recognition systems. Contextual features to be examined include whether users are drawing or viewing a shape, whether users are viewing the beautified or hand-drawn shape, accompanying hand movements, domain knowledge, and accompanying shapes in the diagram. User studies in perception will determine how geometric features co-vary and how shapes should be varied to agree with human perception.
Graphical diagrams are an important part of the educational process. Unfortunately, they are time-consuming to correct and are usually omitted from the testing process despite evidence that testing aids in learning of subject material. Sketch recognition systems can be built to recognize hand-drawn diagrams, but they currently take a long time to build and require expertise in sketch recognition. This project has the potential to provide foundational work that could lead to the development of a tool to allow instructors, without sketch recognition expertise, to build their own sketch recognition tools. Further, this project proposes to build geometric primitive and constraint recognizers based on perception and context to make the creation of sketch recognition systems more intuitive for non-experts in sketch recognition by better matching computer-based recognition to perceptual and contextual expectations. The results from this project will be implemented in the LADDER/GUILD technologies to 1) improve recognition results, making the sketch recognition systems more useful for instructors, and 2) improve automatic generation of shape descriptions to simplify sketch system creation, making it more practical for instructors to use the system.
