Observational studies are relatively inexpensive, but often flawed, substitutes for randomized experiments to examine the causal effect of a treatment. An observational study may be flawed because, before treatment, the observed treated group may not have been comparable to the untreated group, which can lead to a biased estimation of a treatment effect. As one example, observational studies suggested hormone replacement therapy prevents heart attacks among postmenopausal women, while randomized trials showed otherwise. Still, on multiple occasions, observational studies have provided strong statistical evidence to support implementation of an intervention, such as when observational studies provided strong evidence that smoking causes lung cancer. In recent years, observational studies have provided evidence that teenage vaping has a serious effect on lung disease which has led to policy interventions to curb teenage vaping. But the strength of observational study evidence is judged largely by informal/semi-formal rules. For example, the evidence is considered stronger when a similar treatment effect is seen across many independently conducted studies. How the rules that are used is not typically transparent during the assessment of the statistical evidence, and thus, how cautious one should be about how solid the evidence is for a causal claim is often not transparent. This project aims to make how strong the evidence is from observational studies more transparent by developing statistical methodologies to formalize some of the existing informal rules on strengthening scientific evidence from observational studies. To increase their accessibility, the PI, with help from a graduate student, will also incorporate, through software, lessons and projects, these methods in courses taught to graduate students from different empirical fields.
This project will develop several methods for expanding the scope of use of evidence factors in observational study designs. An evidence factors analysis builds statistically independent pieces of evidence (called evidence factors) which, if vulnerable, are vulnerable differently to potential biases. The PI will develop methodologies for evidence factors analysis in novel study designs, such as event studies. The quality of a design and an analysis of a study will be evaluated by statistical power and design sensitivity. The scope of evidence factors is limited if considered only under existing study designs. This grant has the long-term goal of developing new and improved observational study designs which incorporate evidence factors analysis. Construction of these designs typically requires solving NP-hard problems. For example, evidence factors can be built in stratified designs, but creating such a design, while controlling for many confounders, requires solving an NP-hard graph partitioning problem. The PI will develop approximation algorithms to solve these design problems using discrete and combinatorial optimization methods. These algorithms will likely also appeal to the applied mathematics community. This project will also develop evidence factors analysis for robust inference in composite studies which combine, in one design and analysis, aspects of different studies.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
