The growing number of treatment choices, as well as the rapid escalation in the cost of health care has spawned the need for scientifically rigorous comparisons of the efficacy and safety of drugs, devices and treatments in clinical practice. Most quantitative comparisons carried out by the Evidence Based Practice Centers (EPCs) funded by AHRQ have relied on meta-analyses comparing two treatments for one outcome. But many of the questions posed to assessors involve multiple treatments and multiple outcomes. Standard assessments have approached these questions separately, leading to a plethora of analyses to interpret and no quantitatively rigorous methods for integrating them. Multivariate statistical methods offer a solution. Statistical methods for simultaneously analyzing multiple treatments or for analyzing multiple outcomes have appeared in the past decade, along with applications to important problems. Because they are new, these methods have not penetrated into the standard clinical research arsenal and remain the provenance of experts. To make them more practical, particularly for groups like EPCs, we propose further development of the methods, empirical assessment of their strengths and weaknesses, integration into publicly available, easy to-use software and application to key clinical questions. In particular, we propose a three-year project to carry out the following three Specific Aims focusing on intervention studies: 1) Develop new and enhance existing techniques for the statistical analysis of possibly incomplete multivariate meta-analytic data involving multiple (>2) treatments, multiple outcomes or categories of outcomes and multiple follow-up times; 2) Implement these methods in an ongoing open-source software project so users will have access to customized and customizable algorithms and support for their use through analytic wizards that will guide users to appropriate statistical methods and presentation quality graphics and reports;and 3) Apply multivariate methods to a set of key clinical questions that we have identified as well as to topics in the EPC program as they arise over the course of this application so as to gain empirical insight into their strengths and weaknesses and to demonstrate their contribution to improving comparative effectiveness reviews. Completion of these three aims will directly benefit the comparative effectiveness research program at AHRQ by providing state-of-the art methods implemented in user-friendly open-source software that will be made freely available to the public. The open-source nature of the software will enable it to incorporate all of the capabilities of the R and OpenBugs statistical programs and will facilitate the incorporation of new methods as they are developed. The user interface will allow the non-statistically expert user access to these features without the need for programming expertise.

Public Health Relevance

Proper evaluation of the safety and effectiveness of clinical care must address the many different treatments available and the many different clinical endpoints that describe patient outcomes. Quantitative syntheses of data from completed studies using meta-analysis have traditionally compared only two treatments for a single outcome. We propose to develop statistical methods and software for collectively analyzing multiple treatments across multiple endpoints and apply them to rank treatments based on multiple dimensions of health and well- being.

National Institute of Health (NIH)
Agency for Healthcare Research and Quality (AHRQ)
Research Project (R01)
Project #
Application #
Study Section
Health Care Technology and Decision Science (HTDS)
Program Officer
Chiang, Yen-Pin
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Tufts University
United States
Zip Code
Bannuru, Raveendhara R; Schmid, Christopher H; Kent, David M et al. (2015) Comparative effectiveness of pharmacologic interventions for knee osteoarthritis: a systematic review and network meta-analysis. Ann Intern Med 162:46-54
Trikalinos, Thomas A; Hoaglin, David C; Schmid, Christopher H (2014) An empirical comparison of univariate and multivariate meta-analyses for categorical outcomes. Stat Med 33:1441-59
Schmid, Christopher H; Trikalinos, Thomas A; Olkin, Ingram (2014) Bayesian network meta-analysis for unordered categorical outcomes with incomplete data. Res Synth Methods 5:162-85
Nikolakopoulou, Adriani; Chaimani, Anna; Veroniki, Areti Angeliki et al. (2014) Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS One 9:e86754
Chaimani, Anna; Vasiliadis, Haris S; Pandis, Nikolaos et al. (2013) Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. Int J Epidemiol 42:1120-31
Terasawa, Teruhiko; Trikalinos, Nikolaos A; Djulbegovic, Benjamin et al. (2013) Comparative efficacy of first-line therapies for advanced-stage chronic lymphocytic leukemia: a multiple-treatment meta-analysis. Cancer Treat Rev 39:340-9
Trikalinos, Thomas A; Olkin, Ingram (2012) Meta-analysis of effect sizes reported at multiple time points: a multivariate approach. Clin Trials 9:610-20
Jansen, Jeroen P; Schmid, Christopher H; Salanti, Georgia (2012) Directed acyclic graphs can help understand bias in indirect and mixed treatment comparisons. J Clin Epidemiol 65:798-807
Bannuru, R R; Natov, N S; Dasi, U R et al. (2011) Therapeutic trajectory following intra-articular hyaluronic acid injection in knee osteoarthritis--meta-analysis. Osteoarthritis Cartilage 19:611-9
Wallace, Byron C; Schmid, Christopher H; Lau, Joseph et al. (2009) Meta-Analyst: software for meta-analysis of binary, continuous and diagnostic data. BMC Med Res Methodol 9:80