This research project deals with a new method, called the short-time Gaussian cell mapping method for solving nonlinear stochastic optimal control problems. Stochastic nonlinear optimal control problems can be found in civil engineering structures subject to earthquake, ocean waves, and wind loading excitations, in precision machines, manufacturing processes and robots, and in armed vehicles (land or air), target tracking systems and guidance systems operated in combat or rough environments. Few effective methods exist for obtaining optimal control solutions of complex nonlinear systems to random excitations. The method has been applied to some very challenging nonlinear oscillators under Gaussian white noise excitations and has been proven to be effective and accurate in analyzing complex nonlinear stochastic systems. This project develops this method further for nonlinear stochastic optimal control problems. The results of this research provide the capability to more efficiently and accurately predict and control the response of nonlinear systems subject to stochastic excitations, to better assess the reliability of the system under harsh random environments, and to develop new strategies for controlling the state-constrained stochastic system.