Robust statistical methods and software for meta-analysis with outlying studies Principal Investigator: Haitao Chu, M.D., Ph.D. Summary The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce unbiased and precise evidence to make health care safer, more accessible and affordable. The rapid growth of interest in comparative effectiveness research (CER) and evidence-based medicine (EBM) has led to a dramatic increase in attention paid to systematic reviews and meta-analyses, which synthesize and contrast multiple studies into a form of evidence that can be used to underpin guidelines, patient decision aids, and other products that shape health care. Outlying studies frequently appear in systematic reviews and meta-analyses even with stringent inclusion and exclusion criteria. However, traditional meta-analysis approaches cannot effectively and efficiently handle outlying studies. Specifically, conventional approaches commonly include two potentially iterative steps: 1) to detect and identify outlying studies; 2) to conduct sensitivity analyses by excluding the outlying studies. This two- step approach may need to be examined iteratively as the decision rule to determine whether a study is an outlier may change after we exclude ?outlying? studies. Furthermore, different outlier detection methods may identify different studies as ?outliers?. Thus, this approach may not be efficient given the ever increasing volume of studies and increasing demand for timely synthesized evidence, and may lead to inconsistent results when the results differ significantly when outlying studies are included versus excluded, or when different outlier detection methods identify different outliers. In response to PA-15-147, the overall goal of this proposal is to develop robust statistical methods to better estimate overall effect size in meta-analyses with a few outlying studies. In the presence of outlying studies, the between-study variance is commonly overestimated. In consequence, the commonly used methods to estimate the overall treatment effect can be sensitive to outlying studies, which in turn will impact the validity and reliability of systematic reviews and meta-analyses. In this proposal, we will focus on developing innovative statistical methods and software to accurately estimate overall mean effect size using penalized-likelihood based methods, which take the advantages of both the conventional fixed-effect and random-effects models. In addition, we propose to empirically examine the performance of the proposed and existing methods by reanalyzing a randomly selected 2,300 meta-analyses published in the Cochrane Database of Systematic Reviews. Furthermore, the performance between the proposed and existing methods will be rigorously investigated using carefully designed extensive simulations. User-friendly and freely available software will be developed for users to implement our proposed methods accurately and easily. The completion of this proposal will substantially advance CER and EBM through innovative meta-analysis methods. It will improve public health by facilitating treatment selection for various cancers, and for cardiovascular, infectious, and other diseases.

Public Health Relevance

The overall goal of this project is to develop innovative and robust statistical methods and software for meta- analysis with outlying studies. Specifically, we will focus on developing innovative statistical methods and corresponding software to accurately estimate overall mean effect size using penalized-likelihood based methods, which take the advantages of both the conventional fixed-effect and random-effects models. The proposed statistical methodology and software will be broadly applicable to meta-analyses with outlying studies. It will improve comparative effectiveness research and public health by facilitating evidence synthesis of various cancers, cardiovascular, infectious and other diseases.

Agency
National Institute of Health (NIH)
Institute
Agency for Healthcare Research and Quality (AHRQ)
Type
Small Research Grants (R03)
Project #
1R03HS024743-01A1
Application #
9312537
Study Section
Healthcare Effectiveness and Outcomes Research (HEOR)
Program Officer
Banez, Lionel L
Project Start
2017-04-01
Project End
2018-03-31
Budget Start
2017-04-01
Budget End
2018-03-31
Support Year
1
Fiscal Year
2017
Total Cost
$97,841
Indirect Cost
Name
University of Minnesota Twin Cities
Department
Biostatistics & Other Math Sci
Type
Schools of Public Health
DUNS #
555917996
City
Minneapolis
State
MN
Country
United States
Zip Code
55455
Ma, Xiaoyue; Lin, Lifeng; Qu, Zhiyong et al. (2018) Performance of Between-study Heterogeneity Measures in the Cochrane Library. Epidemiology 29:821-824
Lin, Lifeng (2018) Quantifying and presenting overall evidence in network meta-analysis. Stat Med 37:4114-4125