Propensity score analysis is a technique for addressing the problem of selection bias in nonequivalent control group designs. It involves estimating each participant's conditional probability of treatment assignment given his or her vector of observed covariates and using these probabilities (i.e., propensity scores) to balance nonequivalent groups using matching, stratification, covariance adjustment (ANCOVA), weighting, or some combination of these adjustment methods. Most applied research using propensity scores has involved large samples, computing propensity scores via logistic regression, and making analytic adjustments after stratifying on the distribution of propensity scores. Considerably less attention has been paid to other methods of computing propensity scores, like classification trees and bootstrap aggregation or "bagging." However, there is some evidence that the method by which propensity scores are computed impacts the estimates of treatment effects obtained using propensity score analysis. This doctoral dissertation research project will use simulated data to determine the relative performance of three methods of computing estimated propensity scores (logistic regression, classification trees, and bagging) crossed with three types of adjustments that use propensity scores (stratification, covariance adjustment, and weighting).
This line of research is important because there are circumstances where it is not feasible for researchers to conduct randomized experiments. In such cases, researchers often resort to quasi-experiments. The problem with quasi-experiments is that the estimates of treatment effects may be biased due to the nonrandom selection process. Researchers must make adjustments to the estimates of treatment effects in an attempt to minimize the selection bias. This study will result in additional guidelines on the use of propensity score methods. Consequently, researchers in various fields may be able to consult the guidelines, employ propensity score methods, and have greater confidence in the accuracy of adjusted estimates of treatment effects from quasi-experiments. As a Doctoral Dissertation Research Improvement award, this award also will provide support to enable a promising student to establish a strong independent research career.
