Numerical methods for controlled nonlinear stochastic delay equations will be developed. The basic approximating scheme is the so-called Markov chain approximation method which is the current method of choice for the non-delay problem. This will be extended to cover the new systems of interest. Basic theorems on qualitative and approximation properties of the underlying systems will be developed, and the long-term behavior analyzed. Numerically efficient algorithms will be developed. Reducing memory requirements is a major task, perhaps the most crucial, since such systems typically require memory that is beyond the capabilities of current computers. There are various data structures, system representations, and dual formulations that are very promising. In addition, we will develop efficient methods of managing networks of mobiles that operate in an environment where the connecting channels are randomly time varying because of the scattering due to the motions of the mobiles. The basic technique will be based on the perturbed Liapunov function method, a powerful approach for stability analysis when the system is not Markovian.
Nonlinear stochastic delay equations are ubiquitous in applications. Of particular interest are applications to high-speed communications. Such systems are always subject to delays, and are stochastic and nonlinear. This has been the subject of much interest, but current methods of control try to get around the effects of the delays with various tricks that lead to conservative (and often unstable) systems. But it is apparent that the use of the full power of optimal stochastic control theory, taking the true delays into account, can greatly improve the operation. This can only be done if numerical approximations are well-understood and efficient algorithms available.
