A common goal in health outcome and policy research, as well as in other disciplines, is to evaluate the possible causal effect of an intervention, which is broadly defined as a policy change, program participation, or a medical treatment. In health outcome research, many datasets are observational since randomization is often not feasible or ethical. Also, many of these datasets are obtained from large probability surveys. This presents major challenges in inferring causal relationships due to the following facts: (1) the differences in outcomes for intervention groups could be due to the differences in covariates prior to the intervention; and (2) they always involve unequal weighting due to complex sampling designs. Propensity score-based adjustments are widely used to reduce the confounding bias of covariates in observational studies. But there is a critical methodological gap with regard to appropriately incorporating complex survey design in propensity score analysis, and there is a significant amount of confusion among researchers regarding the best practice for interpreting causal effect when using survey data. The overarching goal of this project is to develop a systematic and statistically valid approach for employing causal inference techniques with complex healthcare survey data. Because of the use of sampling weights, the proposed methods are particularly useful in the presence of heterogeneous treatment effects, i.e. different groups may respond differently to a policy change or the introduction of a certain medical treatment. The proposed study will achieve four specific aims: (1) Develop a potential-outcome-based theoretical framework to streamline causal inference in complex surveys; (2) Develop both propensity score and survey design adjusted estimators, including weighted, stratified and matched estimators; (3) Conduct extensive simulation studies to evaluate the performance of various estimators under different practical scenarios and develop a statistical software package for practitioners; and (4) Apply the proposed methodology to a real survey for comparative trauma care research. This study is expected to fill a critical gap in healthcare policy and treatment effect evaluation research by extending the commonly used propensity score adjustment for non-survey data to complex sampling designs. Findings of this study will help promote AHRQ's mission to produce more accurate evidence for health care program evaluation and to improve the current practice of comparative health outcome research. A significant contribution is this general purpose methodology which will be widely applicable and can benefit government agencies, policy makers, and social, political and health science researchers, in those situations where survey data are vital sources for comparative outcomes research and program policy evaluation.

Public Health Relevance

The proposed research will develop a systematic and statistically valid approach for healthcare research, by combining causal inference techniques with complex healthcare survey data. Healthcare survey data such as the Healthcare Cost and Utilization Project (HCUP) are major sources of information for healthcare research and program evaluation. But there is a significant amount of confusion in current practice on how to combine causal inference methods with survey data. The project will address this key methodology gap by providing a formal statistical framework and corresponding estimation strategies for identifying causal treatment effects using complex healthcare survey data. We will apply these methods to the Nationwide Emergency Department Sample (NEDS) to evaluate the impact of level of trauma center care on emergency department mortality at the population level and to consider how the insurance status may modify the undertriage effect. We envision that this general purpose methodology will be widely applicable and can benefit government agencies, policy makers, and health science researchers, where survey data are the vital sources for comparative outcomes research and program policy evaluation.

National Institute of Health (NIH)
Agency for Healthcare Research and Quality (AHRQ)
Research Project (R01)
Project #
Application #
Study Section
Healthcare Effectiveness and Outcomes Research (HEOR)
Program Officer
Taylor, Amy K
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Ohio State University
Biostatistics & Other Math Sci
Schools of Public Health
United States
Zip Code
Lu, Bo; Cai, Dingjiao; Tong, Xingwei (2018) Testing causal effects in observational survival data using propensity score matching design. Stat Med 37:1846-1858
Wheeler, Krista K; Shi, Junxin; Xiang, Henry et al. (2018) US pediatric trauma patient unplanned 30-day readmissions. J Pediatr Surg 53:765-770
Shi, Junxin; Shen, Jiabin; Caupp, Sarah et al. (2018) A new weighted injury severity scoring system: Better predictive power for pediatric trauma mortality. J Trauma Acute Care Surg 85:334-340
Chen, Cheng; Peng, Jin; Sribnick, Eric A et al. (2018) Trend of Age-Adjusted Rates of Pediatric Traumatic Brain Injury in U.S. Emergency Departments from 2006 to 2013. Int J Environ Res Public Health 15:
Wheeler, Krista K; Shi, Junxin; Nordin, Andrew B et al. (2018) U.S. Pediatric Burn Patient 30-Day Readmissions. J Burn Care Res 39:73-81
Corrado, Michelle M; Shi, Junxin; Wheeler, Krista K et al. (2017) Emergency medical services (EMS) versus non-EMS transport among injured children in the United States. Am J Emerg Med 35:475-478
Peng, Jin; Wheeler, Krista; Groner, Jonathan I et al. (2017) Undertriage of Pediatric Major Trauma Patients in the United States. Clin Pediatr (Phila) 56:845-853
Chen, Cheng; Shi, Junxin; Stanley, Rachel M et al. (2017) U.S. Trends of ED Visits for Pediatric Traumatic Brain Injuries: Implications for Clinical Trials. Int J Environ Res Public Health 14:
Shi, Junxin; Lu, Bo; Wheeler, Krista K et al. (2016) Unmeasured Confounding in Observational Studies with Multiple Treatment Arms: Comparing Emergency Department Mortality of Severe Trauma Patients by Trauma Center Level. Epidemiology 27:624-32