This proposal is concerned with asymptotic and small sample inference related to random recursive equations and associated large deviation problems. Random recursive equations, also referred to as stochastic fixed point equations (SFPE), arise in several areas of contemporary science, including: (i) cell-biology, (ii) analysis of algorithms, (iii) financial time-series modeling, (iv) study of perpetuities, (v) actuarial science, (vi) risk management, (vii) ranking of web-pages, and (viii) processes on complex networks. This proposal is concerned with developing new statistical methods that integrate, refine, and sharpen ideas from large deviation theory, semi-parametric and non-parametric inference, efficient importance sampling. It addresses the following basic questions: (1) How to obtain confidence and prediction intervals for the tail probability in risk models that integrate complex financial and insurance processes? (2) How to efficiently estimate the page-ranks of web-pages and understand the factors that influence them? (3) How to provide statistical comparisons between running times of random recursive algorithms? The answer to question (1) is of significant interest to researchers in actuarial science and risk management. The answer to question (2) will enable the development of policies for better utilization of resources. The answer to (3) will yield quantitative methods for developing and comparing recursive algorithms which are used, for example, in computer science.
Random recursive equations allow one to unify a wide class of problems that arise in scientific investigations. In a range of applications, scientists are often interested in understanding the probabilities of occurrence of very rare event, which could nevertheless have catastrophic consequences. These rare events could be rare types of cancer whose prevalence rate is small, or the probability of bankruptcy of a financial institution, or beginning stages of resistance to a drug. A key issue is that, while extensive amounts of data are available to model and analyze the frequently occurring events, the amount of data available to study these rare events is perennially low, making the inferential problem challenging. This proposal is concerned with mathematical, statistical, and computational methods to address these challenging issues and provide concrete answers to some of the problems concerning probabilities of rare events.
This project was concerned with modeling, simulation, and statistical analyses of rare events. These problems are critical since, rare events, when they occur, are disastrous for the society. A typical example is the financial crisis and a more recent one is the spread of Ebola. While many problems involve huge amounts of data, there are not many statistical methods available whenthe data are sparse. In these situations simulating the rare events is a useful strategy. The models studied in this project were of very general nature and can be applied to various including actuarial science, cascades of failures in cloud computing, beginning stages of drug resistance, and financial risk. The contributions from this project were multifold: first, we were able to find new algorithms that facilitate simulating the rare events. It should be noted that it may take several months, if not years, of computing to see one rare event without our proposed algorithms. The algorithms that we discovered have been rigorously proven to accomplish the task. Second, our method allows us to develop a unified inferential framework for a wide variety of similar problems involving rare events. Third, the methods can be applied to various real-time decision making problems. As a part of the grant, we wrote foundational papers developing new mathematical and statistical aspects for analysis of random recursive equations. These papers appeared in top-tiered journals in statistics and probability. As an out-reach to decision makers, we have several papers in journals focusssing in Operations Research. As a broader impact, several papers were written in parasitology where the rare event is concerned with emerging drug resistance. Also, as a part of the grant we have new statistical methodologies for the analyses of rare event data. These methodologies are theoretically justified and can play a critical role in identifying emerging rare-events when data are sparse. Finally all software produced towards the project are publically available. Year 1 work: Development of new approach to solving random recursive equations and development of a variety of examples. Year 2 Work: Development of Algorithms for the above models for rare event simulation and extending the scope even further. Year3 Work: Outreach to OR community; new examples and ideas and further extension developed. Statistical methods for rrare events using very sparse data. On going work: Detailed statistical analysis for sparse data using the methodologies developed in the above grant. Total Number of papers Published in peer-reviewed journal and conferences: Subject matter of the grant: 7 on Related Issues: 6 Additional work from this grant (technical report ): 3 Total outpout : 16
