Risk prediction is inherent to all clinical practice and public health and has been a topic of scientific research for decades. Formal prediction models are frequently used to enhance clinicians' and researchers' ability to quantify and communicate risk. However, a prediction model is only useful if it is accurate when applied outside of the population within which it was developed. Unfortunately, many prediction models in use today prove inaccurate when applied over time and to new populations, yielding not only inaccurate predictions but also a false level of confidence about the quality of their risk assessments. This commonly occurs because models are applied to patients with different clinical characteristics and risk of disease, to medical practices that differ from those used to develop the model, and to methods of care that constantly change over time. The current scientific paradigm does not readily allow models to accommodate these differences. As a result, model accuracy is often compromised for years of clinical use, new models are slow to be developed (if at all), and these new models are no better able to account for changing patient populations or medical practice than the original models. A potential solution to these problems is `Dynamic Prediction Modeling.' Rather than using existing models in practice without accommodating their inevitable degradation in performance and, at best, infrequently developing new models with the same limitations, dynamic prediction modeling updates an existing prediction model continually as new data are accrued. In this approach, the updated models combine the information that is captured in the original model with data from new patients to produce an updated model for future predictions. As a result of this ongoing model-refinement process, dynamic prediction models have the potential to enhance and maintain model accuracy in the presence of changing patient populations and medical practices over time. Our objective in this proposal is to develop and test this new paradigm for risk prediction through rigorous statistical and applied research, to provide comprehensive guidance for the real-world use of dynamic prediction modeling, and thus to remove critical barriers to the wider dissemination of these methods in clinical research and practice. Specifically, this project will: (1) use formal and comprehensive simulations to develop guidelines for implementing dynamic model recalibration, revision, and extension; (2) test and compare these dynamic prediction modeling approaches with the traditional approach to prediction modeling in two real world and diverse clinical settings, and then refine the methods to enhance accuracy and generalizability; and (3) formally and prospectively test the implementation of dynamic prediction modeling in a large, multicenter population of intensive care unit patients to demonstrate the utility, feasibility, and accuracy of dynamic prediction modeling methods in a real-world setting. The ultimate goal is to enhance the generalizability and usefulness of prediction models and improve our ability to deliver precision care.
The ability to quantify the risk that a patient or population will develop a disease, respond to therapy, or have a clinical outcome is of tantamount importance for delivering high-quality patient care and improving public health. Although scientific methods exist to develop prediction models to accomplish these goals, these models typically do not work when applied over time and clinical settings. We will advance a new paradigm for risk prediction, known as dynamic prediction modeling, through rigorous statistical and applied research, to ensure that risk- prediction models are useful and accurate when used in practice, enhance the delivery of precision care, and improve the public's health.
