In the U.S., more than 600,000 knee osteoarthritis (OA)-related total knee joint replacement (TKR) cases are reported every year, exceeding $17 billion estimated direct costs annually. There is a growing need for disease- modifying therapies that prevent or delay the need for TKR. However, development of such therapies remains challenging due to the lack of objective and measurable OA biomarkers for disease progression. The course of the OA is highly variable between individuals and the OA progresses too slowly, making it difficult to identify sensitive OA biomarkers capable of capturing minor changes on the knee joint. This has slowed development of effective therapies and prevents physicians from providing the most effective advice about minimizing the need for TKR. In this project, our goal is to develop imaging biomarkers to monitor minor OA-related changes in knee joint health that lead to TKR. To achieve this goal, we will combine novel deep learning algorithms with clinical and imaging data from the Osteoarthritis Initiative (OAI). The OAI dataset includes clinical data, biospecimens, radiographs, and magnetic resonance (MR) images collected over 8 years. The proposed project has three Specific Aims: (i) to develop an automated OA-relevant biomarker identification tool from the bilateral posteroanterior fixed-flexion knee radiographs using deep convolutional neural networks (CNNs) and recurrent neural networks (RNNs) combined with the OA progression outcome of subjects (n = 882); (ii) to develop an automated OA-relevant biomarker identification tool from structural and compositional MR images using 3D CNNs with RNNs combined with the OA progression outcome of subjects (n = 882); and (iii) to determine whether deep learning?based imaging biomarkers can act as surrogates to predict the OA progression using a subject cohort (n = 296) independent of the cohort used to identify imaging biomarkers. The proposed project will couple deep learning with diagnostic radiology to unveil key combinations of OA-relevant features directly from images with minimal user interaction. This will facilitate fast individualized assessment of OA progression using whole knee joint images directly. If successful, this study will bring new insights into the development of imaging biomarkers for OA progression and more broadly into our understanding and treatment of OA. The knowledge gained in this project will help to advance close monitoring of OA progression by opening new perspectives on the regions and parameters for potential inclusion in both intervention studies and clinical practice.

Public Health Relevance

Osteoarthritis (OA) is a chronic degenerative disorder of joints and is the most common reason leading to total knee joint replacement. Our proposed study aims to develop a novel automated OA-relevant imaging biomarker identification system based on radiographs, magnetic resonance images, and deep learning methods to study knee OA progression. We will address whether combining deep learning algorithms with medical images will determine the key combinations of features relevant to knee OA that are required to accurately predict the OA progression outcome.

National Institute of Health (NIH)
National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS)
Research Project (R01)
Project #
Application #
Study Section
Emerging Imaging Technologies and Applications Study Section (EITA)
Program Officer
Zheng, Xincheng
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
New York University
Schools of Medicine
New York
United States
Zip Code