People are talking to dialog systems in everyday life. Siri, Alexa and others have become household names. But as any user of these systems knows, they are far from perfect. They are also currently limited in terms of the types of things they can do. Usage data is essential to improve these systems using machine learning and artificial intelligence techniques, but companies that create dialogue systems often keep this data to themselves, making it harder for researchers to create, improve, and evaluate such systems properly. DialPort was created with the goal of gathering data from real users for dialog systems around the world. Researchers can connect their systems to the Portal or request the data that others have collected. Beyond this, for researchers who need help creating a dialog system that they could use to run studies and collect data, the DialPort DialogEcosystem provides them with access to tools for the creation of their systems and tutorials on how to use them. For researchers who already have systems and want to test them using human computation (often called crowdsourcing), the DialPort DialogEcosystem provides easy task creation and connection to major crowdsourcing sites. And to lower the barrier to entry to the field, the DialPort DialogEcosystem helps train young students with its REAL Challenge in which students can imagine ideal dialog systems and learn how to create them. The results from this project will ultimately impact every person who uses dialog systems in daily life.
The DialPort project, previously funded by the Computer and Information Science and Engineering (CISE) Research Infrastructure Program, has given the Spoken Dialog Community access to tools, data and users. Researchers who want to create a new dialog system consult DialPort's website for access to the tools they need. When a dialog system is up and running, they connect it to the Portal to get real users to communicate with their systems. The DialPort DialogEcosystem will keep up with the evolving needs of a growing community in several ways. More tools and tutorials will be available. Real user data has started to flow to the systems connected to the Portal and that flow will increase substantially in order to produce the large amounts of data that are needed by state-of-the-art systems. Many researchers test early versions of their systems with crowdworkers, but they are not familiar with how to set up tasks and run quality control and they need help. DialPort DialogEcosystem will respond to these evolving needs with its DialCrowd tools. The field of researchers who are creating dialog systems is expanding. The field of Question Answering is now using chatbots as a means of testing their retrieval capabilities. In Machine Learning, researchers are working on natural language generation and finding that dialog systems are vehicles that can test their work. The DialogEcosystem will reach out to everyone who works on dialog systems and offer a complete framework that can serve their needs from creation to assessment. Beyond present researchers, the DialPort DialogEcosystem will reach out to young students to teach them about our field. The DialPort DialogEcosystem will: create a handheld version of the Portal to address how real users and workers access the Portal; greatly extend DialTools to include wrappers and tutorials for popular system creation tools, extend DialCrowd to help researchers create, assess and analyze results from Crowdwork tasks, create a new REAL Challenge to help high school and undergraduate students become familiar with our field.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
