This is the first year of a three year continuing grant. As computing moves in the direction of small portable devices, speech and pen will become input modalities of choice. This research addresses a critical issue of the identification of the structural, linguistic and performance characteristics of handwriting and speech and issues of their interaction when used together in an application. A series of subject-placed experiments will assess writing and speech when they are: (1) channeled by different types of system prompts, (2) resolving typical system recognition errors, and (3) experiencing system response delays deferring in magnitude and predictability. This research will provide a more complete understanding of how people will write and speak to future interactive systems and how this can be altered by fundamental parameters of system interactivity. It will also provide a more principled basis for the design of complex multimodal systems.//
