This project examines the impact of animated or conversational agent technology on web-based surveys. Animated agents are anthropomorphic software objects in the user interface that, in their most advanced implementations, produce gestures, display facial expressions, and move their eyes in coordination with their speech. They have been shown to improve user performance in some task domains (e.g. tutoring). When might this technology help or hurt survey data quality and respondents' satisfaction? How sophisticated must the agents be in order to provide benefit--or harm? In a web survey, an agent might motivate respondents to participate and complete the task, much like human interviewers do, and help respondents understand the survey questions as intended while allowing respondents to participate at their convenience, as in ordinary web surveys. On the other hand, the presence of an agent might discourage honest responding to questions about sensitive topics, much as human interviewers have been shown to do. In this project a series of laboratory experiments examine more and less human-like agents that ask questions about respondents' sensitive and non-sensitive behaviors. The studies contrast data quality and user satisfaction in non-agent web surveys to those with interface agents that vary in their dialogue capability, the degree to which they provide visual and spoken cues about their internal states, and the degree of intentionality in their speech. The agents are simulated with software that converts a video image of a live interviewer into an animation in real time; respondents thus believe they are interacting with a computer-generated agent even though there is actually a human behind the "agent." In the experiments about non-sensitive behaviors, respondents answer on the basis of fictional scenarios so that the accuracy of their answers can be determined. One possible outcome is that agents with greater dialogue capability will promote interactions that lead to more accurate understanding and thus more accurate answers. In the experiments about sensitive behaviors, respondents answer about their own lives; more reports of sensitive behaviors indicate greater respondent candor. One possible outcome is that agents with more movement (lips, eyes, and eyebrows) will lead respondents to feel less private and therefore to answer less candidly than with agents whose movement is limited. Respondents' satisfaction is measured with a post-interview questionnaire; the impact of different agent features on how respondents communicate with the survey system is measured by detailed turn-by-turn coding of all dialogue.
The practical impact of this work will be more informed decisions by survey researchers in adopting agent technology. Knowing when agents help and what features help the most can focus decisions about what interfaces to develop and which ones not to develop. For example, if dialogue capability is more important to data quality and user satisfaction than other agent features, this could focus future development efforts on conversational competence of agents more than on visual realism. The theoretical impact of the proposed work will be in two areas. First, it will deepen our understanding of how verbal and non-verbal communication are interconnected, for example how verbal interaction is affected by the fidelity of the agents' facial display. Second, by comparing human-computer and human-human interaction the project will advance knowledge of how attributions of intentionality and human agency affect interaction more generally. This research is supported by the Methodology, Measurement, and Statistics Program and a consortium of federal statistical agencies as part of a joint activity to support research on survey and statistical methodology.
