Virtual assistants, and more generally linguistic user interfaces, will become the norm for mobile and ubiquitous computing. This research aims to create the best open virtual assistant designed to respect privacy. Instead of just simple commands, virtual assistants will be able to perform complex tasks connecting different Internet-of-Things devices and web services. Also, users may decide who, what, when, and how their data are to be shared. By making the technology open-source, this research helps create a competitive industry that offers a great variety of innovative products, instead of closed platform monopolies.
This project unifies all the internet services and "Internet of Things" (IoT) devices into an interoperable web, with an open, crowdsourced, universal encyclopedia of public application interfaces called Thingpedia. Resources in Thingpedia can be connected together using ThingTalk, a high-level virtual assistant language. Another key contribution will be the Linguistic User Interface Network (LUInet) that can understand how to operate the world's digital interfaces in natural language. LUInet uses deep learning to translate natural language into ThingTalk. Privacy with fine-grain access control is provided through open-source federated virtual assistants. Transparent third-party sharing is supported by keeping human-understandable contracts and data transactions with a scalable blockchain technology.
This research contributes to the creation of a decentralized computing ecosystem that protects user privacy and promotes open competition. Natural-language programming expands the utility of computing to ordinary people, reducing the programming bottleneck. All the technologies developed in this project will be made available as open source, supporting further research and development by academia and industry. Thingpedia and the ThingTalk dataset will be an important contribution to natural language processing. The large-scale research program for college and high-school students, with a focus on diverse students, broadens participation and teaches technology, research, and the importance of privacy. All the information related to this project, papers, data, code, and results, are available at http://oval.cs.stanford.edu until at least 2026.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
