The project studies the multi-stream approach to automatic speech recognition (ASR) using stream definitions in the modulation spectrum domain. The multi-stream approach is a new acoustic modeling technique to increase the robustness of ASR systems. It derives different information sub-streams from the speech signal followed by independent feature extraction and probability estimation in each sub-stream. The vectors of class-conditional probabilities obtained from each sub-stream are merged to obtain the final decision. The multi-band model which utilizes different regions of the frequency spectrum in the sub-streams has been shown to be robust to frequency-selective degradation. The aim of the project is to utilize different ranges of the modulation spectrum in different sub-streams. A study of the relative importance of the different ranges of the modulation spectrum indicates that they might carry different information. This makes the various elements of the modulation spectrum interesting candidates for the multi-stream approach. The computation of these features requires looking at longer segments (typically syllable length segments) of the signal. The proposed project is an attempt to apply the new assumptions of relative independence of elements of the feature vector and medium-term time dependencies between feature vectors to ASR with a view to improve its robustness.
