Modern computers are getting remarkably good at producing and understanding human language. But do they accomplish this in the same way that humans do? To address these questions, the investigators will derive measures of the difficulty of sentence comprehension by computer systems that are based on deep-learning technology, a technology that increasingly powers applications such as automatic translation and speech recognition systems. They will then use eye-tracking technology to compare the difficulty that people experience when reading sentences that are temporarily misleading, such as "the horse raced past the barn fell," with the difficulty encountered by the deep-learning systems. Based on this comparison, the researchers will modify the computer models to make them behave more like humans when processing language. This will enhance our understanding of the strategies that humans use to understand sentences while also having the potential to advance language processing technologies.
The eye-tracking-while-reading measurements collected over the course of the project will be accessible to all in an open repository called the Garden Path Benchmark. This benchmark will combine the focus on syntactically challenging sentences traditionally used in psycholinguistics experiments with more recent ‘big data’ approaches to data collection and analysis. The resulting database will contain enough eye-tracking data to get clear estimates of the word-by-word processing difficulty associated with a range of constructions and specific sentences. This will allow researchers to test the quantitative predictions of deep-learning systems and other computational models at a scale that has previously not been possible. The dataset will also be used to develop parsing models that integrate contemporary deep-learning architectures with traditional symbolic parsing models from the psycholinguistics literature. This fusion will make it possible to incorporate scientific assumptions about human cognitive processes, such as reanalysis (the revision of the interpretation of a sentence when it turns out that the reader’s first interpretation was incorrect), into the neural networks. Both the Garden Path Benchmark and the models developed will be released as open access to other researchers, to support further efforts to align machine learning models and human language processing models.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
