Advances in spoken dialogue systems rely on studies of how some strategic system change affects real user behavior. Since any change to a dialogue system provokes a change in what users say, it is probable that non-live, pre-recorded dialog corpora cannot measure what that change will be. This feasibility study, Let's Go Lab, explores the alternative possibility of using a common dialogue platform. The Let's Go system is being made available as a proof-of-concept speech research community resource for dialogue studies. Let's Go answers the phone for the Port Authority of Allegheny County to give scheduling information from 7pm to 6am every day to an average of 60 callers. Researchers propose a study and work with the Let's Go Lab team to modify the base system as needed, run offline tests, and finally "go live", testing their hypotheses on significant quantities of dialogues with real users. The project will explore whether both minor changes (replacing one system utterance) and major ones (changing the dialogue management architecture) are feasible. The Let's Go team is also examining how to lessen startup costs for each new study by reusing past modules, refining "go live" criteria, and training researchers to participate in the preparatory work. If this project is successful, it will pave the way for a new, powerful community resource where results can be compared on a common platform, statistically significant numbers of real users can be tested and new researchers can enter the field easily.
