This project obtains new fundamental knowledge about how people react to errors made by affect-aware technologies. Such technologies analyze measurements such as heart rate, brain activity and body gestures to obtain an estimate of a user's mental and emotional state; they then take actions to improve the user's state - for example, by helping with a task. However, since the measurements are often hard to interpret, affect-aware technologies often make mistakes. Led by an interdisciplinary group of researchers in engineering and psychology, this project will examine how users react to and compensate for different types of errors made by affect-aware technologies. This will help guide future design of such technologies, as it will help researchers and developers identify what the minimal acceptable accuracy of an affect-aware device is and what types of errors most critically need to be reduced by developers. Results of the research will advance national health and well-being in many ways, as affect-aware technologies are becoming increasingly common for diverse applications such as detecting drowsiness in drivers, adaptive automation in flight and resource management, adaptation of learning material to students, and adaptation of rehabilitation exercises to patients. The team will develop new interdisciplinary courses in human factors and human-computer interaction, and will perform outreach about cyber-human systems to multiple groups including K-12 and community college students and teachers all around Wyoming.
The project is structured as a series of four lab studies involving human subjects, all using a set of physiological sensors and the NASA Multi-Attribute Task Battery. As little is known about user reactions to machine errors in affect-aware cyber-human systems, the first three lab studies will systematically vary four critical characteristics: the accuracy with which they recognize the user's psychological state, the magnitude of the actions (changes to task difficulty) taken by the system, the impact that an error has on task performance, and the transparency of the system's decision-making process. The errors will be induced with a Wizard of Oz experiment design in which, unknown to the subject, the machine responses are actually simulated by a human operator. In this project, the user will be asked how they would like to change the difficulty of the Multi-Attribute Task Battery, and errors will be induced by doing the opposite of what the user wants. Users will be unaware of this manipulation, and will be told that the errors are actually due to poor signal processing and pattern recognition. The last study will then examine the trade-off between a system's state recognition accuracy and its user-friendliness with regard to user acceptance of the system. In all four studies, the outcome measures will be objective task performance as well as subjective user experience reported with the NASA Task Load Index and Intrinsic Motivation Inventory. This will provide the research community with detailed information about how different characteristics of affect-aware cyber-human systems influence both objective and subjective aspects of users' experiences with such systems.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
