This project combines local numerical optimization routines from applied mathematics with flexible global models from statistics to create efficient hybrid optimization routines. These routines can address difficult optimization problems involving multi-dimensional and multi-modal functions. We focus on pattern search as our local routine and treed Gaussian processes as our statistical emulator. We develop an implementation in an asynchronous parallel computing environment, and allow for both known and hidden constraints. We also explore optimization of stochastic functions and optimization under uncertainty, as well as robust optimization.
This project solves difficult optimization problems by combining tools from the fields of applied mathematics and statistics. By intertwining efficient local numerical routines with global statistical emulators, we develop efficient and robust algorithms for maximizing or minimizing functions. This inherently inter-disciplinary work has immediate applications in a wide range of fields, and we demonstrate its effectiveness on problems from electrical engineering and hydrology.
Many scientific problems require a complex optimization as part of the solution to the problem, as we need to find a minimum or maximum. Much of the existing research on optimization focuses on local numerical methods, which are efficient at finding a local optimum, but may not look broadly across the whole space, and thus may not find the global optimum. This project combined local numerical methods with statistical modeling. The statistical model can look broadly and suggest areas of interest, and then a local numerical method can be used to quickly find the optimum in each area of interest. By combining these two approaches, we can improve the scope and the efficiency for global optimization. There are many applications for hybrid optimization. This project focused on computer simulation applications, those where the computer represents a situation which is difficult to create or experiment with in practice, such as remediation of contaminated groundwater, creating an automated flight controller that can stabilize unexpected damage to an aircraft, or understanding how people make health care insurance decisions under different policies and laws. These computer simulators are typically expensive to run, and the relationship between inputs and outputs is complex and difficult to understand. Thus statistical tools are helpful for modeling the simulator, and hybrid optimization methods are critical for efficient exploration of the space to find the global optimum. The complex nature of the simulator means that there are typically many local solutions, and it can be challenging to find the global optimum, so the broader view of the statistical model is critical in guiding the local numerical optimization methods. This project has shown the value of combining statistical modles with numerical methods, and it has lead to deeper understanding of problems in hydrology, aeronautical engineering, and policy.
