Better information about the benefits of interventions in individuals has enormous potential to improve clinical decision making. Yet cost effectiveness analyses (CEA) are almost always based on average incremental cost and average incremental benefits found in groups. Since health care resources are allocated by decisions made by and for individual patients, use of average cost effectiveness (CE) ratios can be inappropriate and misleading. Interventions that are cost effective on average may not be cost effective for many (even for most) patients with the index condition and-conversely-interventions that are nominally cost-ineffective may be highly worthwhile in some. Concerns about the inappropriateness of applying average population CE ratios parallel concerns about what treatment is best for an individual patient based on summary results of clinical trials. Our prior work using risk models has shown that substantial differences in baseline risk are ubiquitous across individuals with the same index condition. This risk heterogeneity gives rise to substantial, and often clinically meaningful, differences in therapeutic benefits--particularly when benefits are considered on the absolute scale, the most relevant measure for clinical decision making and CEA. Clinical Prediction Models (CPMs) can be used across research and practice domains to address this risk heterogeneity and are abundant in the literature. Despite the important concerns about the use of average effects and average CE ratios and the availability of CPMs, the potential health and economic impact of better individualization of risk information on clinical decisions remains largely unexamined. Further, just as CEAs typically ignore the potential for population risk stratification, traditional measures used to evaluate CPM and novel risk biomarkers typically ignore the decisional context in which the predictions are applied, and focus instead on """"""""utility-free"""""""" measures of statistical accuracy. Not surprisingly, these measures often poorly anticipate the ultimate clinical usefulness of the predictive information. Thus, our specific aims are:
Aim 1 : To examine the expected value of a risk-based approach to individualizing care and cost effectiveness across a broad range of medical interventions;
Aim 2 : To develop and test appropriate methods to assess prediction models, and incremental improvements in risk prediction, based on a decision analytic framework that estimates the health and economic impact of improved individualized medical decision-making;
Aim 3 : To explore the policy implications of using a risk-based approach to individualize care by: (a) simulating the impact of incentive-based programs, and (b) engaging stakeholders on real-world implementation. This project will: 1) elucidate the overall value of targeting therapy using a risk-based approach;2) help us understand the circumstances in which such an approach might be especially useful;3) provide heuristics and tools to expedite the evaluation of CPMs and novel risk biomarkers; and 4) help us understand how best to incentivize their translation into clinical practice.

Public Health Relevance

The proposed research explores the value of providing to clinicians and patients information regarding each patient's individualized risk of having bad health outcomes so that clinicians can better tailor care. It also aims to develop new tools and methods to help recognize when a new risk factor (e.g., a new biomarker) may provide clinically useful information, and when it is unlikely to do so. Finally, we will explore the potential for improving health through doctor and patient incentives to make decisions those better match patient risks, values and preferences.

National Institute of Health (NIH)
National Institute of Neurological Disorders and Stroke (NINDS)
Research Project--Cooperative Agreements (U01)
Project #
Application #
Study Section
Special Emphasis Panel (ZRG1-HDM-W (52))
Program Officer
Hirtz, Deborah G
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Tufts University
United States
Zip Code
Cohen, Joshua T; Neumann, Peter J; Wong, John B (2017) A Call for Open-Source Cost-Effectiveness Analysis. Ann Intern Med 167:432-433
Costa, Francesco; van Klaveren, David; James, Stefan et al. (2017) Derivation and validation of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy (PRECISE-DAPT) score: a pooled analysis of individual-patient datasets from clinical trials. Lancet 389:1025-1034
Paulus, Jessica K; Kent, David M (2017) Race and Ethnicity: A Part of the Equation for Personalized Clinical Decision Making? Circ Cardiovasc Qual Outcomes 10:
van Klaveren, David; Steyerberg, Ewout W; Serruys, Patrick W et al. (2017) The proposed 'concordance-statistic for benefit' provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol :
Austin, Peter C; van Klaveren, David; Vergouwe, Yvonne et al. (2017) Validation of prediction models: examining temporal and geographic stability of baseline risk and estimated covariate effects. Diagn Progn Res 1:12
van Klaveren, David; Wong, John B; Kent, David M et al. (2017) Biases in Individualized Cost-effectiveness Analysis: Influence of Choices in Modeling Short-Term, Trial-Based, Mortality Risk Reduction and Post-Trial Life Expectancy. Med Decis Making 37:770-778
Olchanski, Natalia; Cohen, Joshua T; Neumann, Peter J et al. (2017) Understanding the Value of Individualized Information: The Impact of Poor Calibration or Discrimination in Outcome Prediction Models. Med Decis Making 37:790-801
Austin, Peter C; Steyerberg, Ewout W (2017) Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat Methods Med Res 26:796-808
Paulus, Jessica K; Lai, Lana Y H; Lundquist, Christine et al. (2016) Field Synopsis of the Role of Sex in Stroke Prediction Models. J Am Heart Assoc 5:
Kent, David M; Nelson, Jason; Dahabreh, Issa J et al. (2016) Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. Int J Epidemiol 45:2075-2088

Showing the most recent 10 out of 24 publications