Pen-based computing has the potential to make a significant improvement in the education of our students in STEM disciplines so as to produce a more skilled technical workforce able to compete in the new global economy. Mathematical sketching, a pen-based interaction paradigm that is providing a new approach to STEM education, is the process of making and exploring dynamic illustrations by associating handwritten mathematics with free-form diagrams. The proposed research will develop new mathematical sketching tools to improve student learning in STEM. High level semantic recognition techniques for understanding mathematical sketches (the combination of handwritten problem description, mathematical expressions, and drawings) will be explored to ensure proper interactive feedback, facilitation of correct cognitive models, and the generation of the most appropriate dynamic visualizations for a given problem. In addition, these techniques will, in part, be used to develop strategies in inferential statics and dynamics, the process of finding appropriate animations when users do not directly specify motion equations in the problem solving process. Given the reliance on recognition-based interaction in mathematical sketching, usability studies will examine how users are affected by recognition feedback and accuracy levels. These studies will provide useful guidelines for ensuring that the mathematical sketching interface is usable by both teacher and student. Two educational studies will also explore how mathematical sketching affects teaching and learning. This work will advance the state-of-the-art in pen-based UIs through the development of techniques and exploration of user interface design issues for enriching mathematical sketching.
Broader Impact: This proposal links research in pen-based UIs with STEM education, a critical application area. Thus, the work has the potential to reach thousands of students each year. The PI has the opportunity to work on this research in conjunction with the University of Central Florida?s EXCEL Program, an NSF STEP-funded project that helps students succeed in math and science during the first two years of college, where the mathematical sketching paradigm is directly applicable. The PI will give demonstrations and lectures through the Florida High Tech Corridor Council techCAMP program to bring mathematical sketching to teachers and students in African American and Hispanic high schools. The proposed research will be integrated into a graduate-level course on pen-based UIs, thus enhancing the education of tomorrow?s pen-based UI designers and researchers. Finally, undergraduate courses will be taught on human computer interaction (HCI) and design of interactive systems that, together with the graduate course, will be an educational foundation for HCI at UCF.
