User interfaces for pen-stylus systems have not managed to deliver the fluid, seamless, natural interaction afforded by pen and paper interfaces, as evidenced by a resurgence of thumb keyboards, soft keyboards, and innovative mappings of numeric keypads to alphanumeric characters. In this project the PI will study the effectiveness of, and explore potential enhancements to, traditional pen/stylus interfaces. The goal is to identify techniques for managing, displaying, and supporting user interaction with ambiguous results while maintaining the fluidity of the pen-based interface. To these ends, the PI will develop advanced AI/pattern recognition models of the information contained by applications, advanced and novel algorithms that use these models to reason intelligently about user intention in interfaces, and prototype applications centered on these algorithms. The PI will test his models in both traditional laboratory and real world settings, including in novel applications to enhance students' learning experiences through incorporation of this research into a deployed smart classroom environment. Major outcomes of the project will be a characterization of when, where, and how intelligence can be incorporated into pen-based interfaces, and a series of specific principles for intelligent interface design in pen systems to guide research in other areas of HCI.
Broader Impacts: The PI's institution is RUI-eligible, and approximately 40% of the students are of Hispanic, African American, Pacific Islander, Filipino, and other non-Asian/Caucasian backgrounds. Furthermore, the PI's department currently enjoys almost no external research funding. Thus, this project, coupled with the planned integration of the work into an educational environment, will expose a large number of students to research experiences and educational opportunities they otherwise would not have.
