The most serious problem with back propagation and other learnng algorithms for training multilayer perceptrons is that the amount of training which is required scales too fast with the size of the multilayer preceptron. The project will attempt to develop a learning algorithm for a neural network architecture with multiple small mulitlayer preceptrons connected in such a way as to exhibit better scaling properties. The P.I. will consider a tree-structured classifier which uses small multilayer preceptrons at its decision nodes to extract nonlinear combinations of features. The first phase of the proposed research will focus on learning algorithms for the multilayer preceptrons at the nodes of the tree. Suitable back propagation-type algorithms will be developed for those purpose. The back propagation algorithm is a generalization of is a gradient descent algorithm frequently used a linear adaptive filtering with a mean-square error criteria. Back propagation-type algorithms will be developed which local minima by applying the author's own work on globally convergent gradient descent algorithms, which is a form of continuous stat simulated annealing. Back propagation-type algorithms will be developed which minimize non-mean-square error criteria by generalizing gradient descent algorithms frequently used for linear adaptive classifiers with a preceptron, relaxation, or minimum probability of error criteria. Back propagation-type algorithms will also be developed which are capable of unsupervised learning. The second phase of the proposed work will focus on the learning algorithm for the overall tree. Here in conjunction with a back propagation- type algorithm training the multilayer perceptrons at the nodes of the tree, we will develop adaptive tree growing and pruning algorithms. The proposed work should provide an important tool for application of neural networks to hard large-scale problems in speech and image recognition, where current best systems are far from equaling human performance.
