Young students who have difficulty in reading fluently can gain confidence by repeatedly reading story passages. Computer reading assessment can help improve student?s fluency while freeing up teacher to help students more efficiently. In previous projects, computer-based technologies have been shown to improve the reading ability of low performing readers in urban schools. Reading verification is a critical task, determining whether a student has correctly read the words on the page, or has mispronounced words, is stuck, or has jumped ahead or back. However, current systems that use automatic speech recognition (ASR) tend towards accepting incorrect pronunciations, instead of figuring out mistakes, and operate poorly when there is noise. This project addresses these issues by studying how to build a reliable computer system that can tell, in real-time, whether students are reading correctly within a noisy classroom, and make it available on many different types of computers. To allow other researchers to compare their studies, the project team will create and make available a dataset of reading examples recorded in noise through the Language Research Lab at the Center of Science and Industry (COSI) museum. By interacting with lab personnel, visitors will be educated on how speech recognition systems work as well as on how language science research is done. Thus, if this project is successful, significantly better computer-based reading assessment technologies will be developed, which will help many children read better, and the public will become better informed about reading, speech technologies and science in general.
The approach investigated in this project abandons the need for full ASR technology, instead performing Reading Verification (RV) directly on the speech signal while jointly performing speech enhancement to focus on the child's speech. The elimination of ASR decoding technology permits creation of smaller verification models that can be deployed on devices or web browsers. Building on pilot work in mispronunciation detection and integration of phonetic information into speech enhancement, a new attention-based tracking approach is examined that directly models phenomena like partial word disfluencies, explores the difference between enhancement for child and adult speech, and investigates a novel joint model of detection and enhancement integrating longitudinal adaptation techniques. The research activities are supported by a new collection of speech data, consisting of stories read in a noisy science museum environment. The recorded dataset will facilitate comparative work in reading assessment and will be released as part of the Ohio Child Speech Corpus.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
