The research objective of this award is for the development of efficient Monte Carlo methods for computing probabilities of rare events in Gaussian random fields and geometric properties of the fields under the conditional distribution given rare events of interest. A crucial methodological idea involves taking advantage of limit theorems and asymptotic approximations in order to guide the construction of efficient Monte Carlo methods that can be applied in the pre-limit. There is a rich literature on asymptotics for the probabilities of certain rare events in Gaussian random fields (such as high level excursions). The PI's exploit the information hidden in the development of such asymptotics in order to develop the Monte Carlo methodology, which in most cases is based on importance sampling. This research also includes the investigation of a class of high dimensional #P-hard problems that the PI's plan to attack by taking advantage of efficient Markov chain Monte Carlo techniques combined with convex optimization algorithms and importance sampling ideas.

This research is motivated by a variety of applications, ranging from environmental sciences, image analysis, statistical applications, risk management and so forth. If successful, the output may highly and positively impact a wide range of scientific areas. For instance, in environmental studies tied to urban development, the ability of efficiently evaluating changes in contamination levels in different areas of a geographic region given that high contamination occurs will add substantial value in the development of policies and decision making processes. The output of this research may also potentially aid other simulation areas dealing with Gaussian random fields and their applications to kriging and optimization.

Project Report

This resarch focuses on the study of efficient stochastic simulation methods for the analysis of complex structures which take as input Gaussian processes and fields. Typically a Gaussian process is a random function which evolves in time. As an example of a Gaussian field one could consider the logarithm of the concentration of ozone in a geographical region - the name field, as opposed to process, is given because the quantity of interest is indexed by spatial coordinates and time, as opposed to only time. The use of Gaussian processes in engineering and scientific applications is broad. This is because Gaussian processes and fields are relatively easy to parameterize and fit (only means, variances, and correlations are required to fit a Gaussian process). However, these objects are used as input to much more complicated models and functions, which actually constitute the output of interest. The scientific work funded by this project has allowed to design and study Monte Carlo algorithms (another name for stochastic simulation) which are the first of their kind: a) We provide the first class of optimal estimators for conditional expectations given high excursions of general Gaussian random fields (i.e. conditional expectations given a extreme value in the maximum value of the field). b) We provide the first class of optimal estimators for extreme value solutions of linear optimization problems with random input. These problems have applications in the evaluation of rare events in transportation networks and systemic risk networks. c) We provide the first class of optimal estimators which allow to estimate without bias steady-state expectations of multidimensional stochastic networks. These models are of significant importance in Operations Research as they can often be used to model the workload of queueing networks. d) We provide the first class of perfect sampling estimators for multidimensional Reflected Brownian Motion (RBM). RBM is one of the corner-stones of modern Operations Research. This model is a non-trivial, highly non-linear, function of a Brownian motion (which is itself an important Gaussian process). RBM can approximate the workload of virtually any queueing network of interest in heavy-traffic. We are able to obtain a simulation estimator which allows to sample exactly, without any bias, from the distribution of RBM at any time - this is the definition of perfect sampling. The results are of significant theoretical importance given that the distribution of RBM is unknown in closed form. We are also able to estimate expectations of the steady-state distribution of RBM without any bias. e) As a part of the development behind d) we developed a new simulation concept, which we call "strong" simulation and which consists in approximating a fully continuous random object with an error which is fully controlled by the modeler with absolute certainty. This project shows that, surprisingly, it is possible to construct "strong" simulation algorithms for a large class of processes and they in fact form the basis for the contributions discussed in items c) and d) above. f) We provide the first closed form asymptotic result for exponential integrals of a general class of smooth Gaussian processes. Furthermore, we provide efficient Monte Carlo estimators for the tail event probabilities. The research funds were able to partially support the research of several PhD students who have finished (or are close to finish) their dissertations. Several of them have graduated and become faculty members in engineering, applied mathematics, and statistics departments accross the country.

Project Start
Project End
Budget Start
2011-07-01
Budget End
2014-06-30
Support Year
Fiscal Year
2010
Total Cost
$300,000
Indirect Cost
Name
Columbia University
Department
Type
DUNS #
City
New York
State
NY
Country
United States
Zip Code
10027