PI Institution: Duke University
The objective of this research is to develop and implement a unified theory for memory and forgetting in artificial neural networks. The novel learning algorithms developed through this research will eliminate interference and catastrophic interference in nonlinear and fully-connected neural networks, thereby enhancing their applicability in a number of engineering applications. The approach is to formulate learning through a constrained backpropagation approach that optimizes the neural network performance subject to long-term memory constraints, which may be deteriorated over time via a penalty function or Lagrange multipliers.
Intellectual Merit The intellectual merit of the proposed research is the development of a novel constrained backpropagation approach that combines constrained optimization theory and classical backpropagation. The newly developed adjoined error gradient and algebraic training formalisms together allow to formulate constrained backpropagation efficiently and effectively, while also exploiting existing artificial neural networks algorithms, such as Levenberg-Marquardt and resilient backpropagation.
Broader Impact The proposed activity will enhance the applicability and effectiveness of on-line adaptive neural networks in a broad spectrum of complex science and engineering problems, namely, function approximation, solution of differential equations, system identification, and control. The constrained-backpropagation theory and algorithms will be implemented on data-assimilation problems, which will benefit society by producing timely predictions about environmental change and dispersion of urban pollutants, and on adaptive dual control, which will produce flight control systems that are fault and damage-tolerant, and make piloted airplanes safer and easier to fly. Also, they will be demonstrated through benchmark problems in robotics and mine hunting using Graphical User Interfaces, for educational and dissemination purposes. This approach has already been proven successful at creating positive synergies and collaborations between Duke University and K-12 students from the Chapel Hill (NC) public schools, as well as small local industries.
