The speech and spoken dialog communities watched with interest as the world discovered Apple's SIRI. One of the first applications of an advanced spoken dialog system to a real world problem, it captured the imagination of potential users; the idea of speaking to an object to obtain information or direct an action has entered into the public mind, and the possibilities seem endless. Yet despite the potential advantages, academia has so far created few spoken dialog systems that serve real users and real applications. In the first three years of being open to the research community, the Let's Go system developed by the PI and her team was used for over 150 publications, including 18 theses outside of its host institution. The speech and spoken dialog communities clearly need more real world systems that furnish data, free architectures and research platforms. The PI's goal in this project is to foster such new real world systems, which will give the speech and spoken dialog communities steady streams of data as well as research platforms that they can use to run studies. The project will engage seasoned researchers (who know what will work and what will not), along with high school and undergraduate students (whose younger minds are free to imagine what speech applications can become, uninfluenced by research results or funding concerns), in an effort to find the next great speech applications.
Broader Impacts: The REAL Challenge will be the spark that ignites the creation of novel real speech applications. The project will address a broad range of students and invigorate research in a way that can be used in other areas of natural language research. It will inspire a new generation of researchers and provide a unique opportunity for young students to work with seasoned researchers. The novel applications that will be found will ultimately be of help to the general public, whether it be for better access to information, for interaction with a robotic helpmate, or perhaps for a new way of communicating with others using social networks.
The goal of the REAL Challenge is to assemble high school and undergraduate students and seasoned researchers in order to find new ways that users can talk to objects. Eventually this will lead to the creation of new applications and thus streams of real user data that the speech and spoken dialog research communities desperately need. The students look for the novel applications and the researchers bring to the table their knowledge of how to implement them. This grant allowed us to prepare for the Challenge and then its Workshop. Our first action was to contact students and their teachers, describing the Challenge. Next was a period during which the team answered their questions and gave examples of how speech is presently used. Then the team contacted colleagues who work in our domain, informing them about the Challenge and its goals and soliciting their participation at the Workshop, at which the students were to present their ideas. The students were helped in proposal writing if they asked for it. Proposals were submitted to the team. Then a group of researchers reviewed the proposals. They looked for feasibility as well as clarity in the description of an idea. The reviewers were asked to give the students generous feedback on their proposals in both of these perspectives. Once the reviews went out, the team started to prepare the students to show their ideas at the Workshop. They all had training on how spoken dialog systems work. They also learned about how to make a poster and how to do a presentation. There were 11 proposals on a variety of subjects. One interesting and unexpected outcome of this Challenge is that as the students proposed interesting applications, they also brought up some major issues concerning the ubiquitous use of speech to communicate with objects that seem to have not been addressed or even acknowledged so far by the research community. One of these issues concerned privacy: the fact that there are many applications that people may not want to use in public (on the bus or walking on the street) since the contents of the communication could be overheard by others in proximity. Another issue is that while researchers have been trying to devise applications that can be used for practically anything by the general public, many of the students saw the use of speech as something that will mostly come into play when there is a highly personalized need (preparing for a talk, cooking a recipe). The proposals came from four countries (US, Ireland, Korea, and China). The US proposals were from high school students and all of the others were from undergrads. Special effort was made to visit the high school students to give them more training about automatically processing speech and about presenting their ideas. We feel that the students, many of whom have never made a presentation before, gained a great life skill from this training. The proposals were: Name(s) School Title So Hyeon Jung Postech Smart Watch BongJin Sohn, JongWoo Choi, DongHyun Kim Sogang University Mobile Cooking App Jung-eun Kim Postech Fashion Advisor Jaichen Shi Shanghai Jiao Tong University Gourmet Keun Woo Park, Jungkook Park KAIST Daily Journaling Yunqi Guo Shanghai Jiao Tong University Human Interaction Chatting System Qizhe Xie Shanghai Jiao Tong University Lecture trainer/Exercise tracker Jude Rosen, Joe Flot Pittsburgh Sci & Tech Academy Skull Zachary McAlexander, David Donehue Pittsburgh Sci & Tech Academy SmartCID Neeloy Chakraborty Pittsburgh Sci & Tech Academy Lead Thou Me On Enno Hermann Trinity College Kitchen Assistant There was constant communication with the students and with their teachers. The Challenge website (dialrc.org/realchallenge) was continually updated with information about the application process and then about preparations for the Workshop. Colleagues could also consult the website to find information about roles they could play (mentoring, reviewing proposals, coming to the Workshop). Our team served as the intermediary between colleagues and the students and their teachers. Some students from countries outside the US were not able to attend the Workshop due to travel costs and to final exams. They prepared for the Workshop with our team by preparing to present via Skype and, if there was a problem with Skype, preparing a pre-recorded video that could be played in front of their poster. We feel that the preparation enabled everyone to arrive at the Workshop well-prepared to present and discuss novel ideas about the use of speech.
