Multi-atlas label fusion (MALF) is a powerful new technology that can automatically detect and label anatomical structures in biomedical images. It is arguably the most successful general-purpose automatic image segmentation technique ever developed. Automatic segmentation is in high demand in clinical and research applications of medical imaging, since segmentation forms a crucial step towards extracting quantitative information from imaging data, and since manual and semi-automatic approaches are ill suited for today's increasingly large and complex imaging datasets. Despite a number of papers that demonstrated outstanding performance of MALF methods across a range of biomedical imaging applications, the broader biomedical imaging research community has been slow to adopt this technique. This can be explained by multiple factors, including the technique's high computational demands, lack of a turnkey software implementation, as well as scarcity of validation in clinical imaging datasets and in the presence of extensive pathology. The present application seeks to remove these barriers and to enable a broad range of clinicians and biomedical researchers to take advantage of MALF technology. It builds on our strong track record of innovation in the MALF field, including a novel redundancy-correcting MALF technique that led in segmentation grand challenges in the past two years.
Aim 1 seeks to improve the computational performance of MALF by replacing dense deformable image registration, by far the most time consuming component of MALF, with faster and less constrained sparse registration strategies. We hypothesize that this will not only reduce the computational cost of MALF, but will also make it more robust to anatomical variability, in particular enabling its use for tumor and lesion segmentation.
Aim 2 proposes algorithmic extensions to MALF that support automatic segmentation of dynamic and multi-modality imaging datasets, which have been largely overlooked in the MALF literature.
Aim 3 will develop a turnkey open-source implementation of MALF methodology. Taking advantage of cloud computing technology, this software will allow users with minimal image processing expertise to take full advantage of MALF segmentation on their desktop.
Aim 3 will also provide a set of publicly available atlases and the means for users to build new custom atlas sets from their own data.
Aim 4 will perform extensive evaluation of the new methods and software in challenging real-world clinical imaging data, including brain and cardiac imaging. As part of this evaluation, we will quantify how well our MALF approach and competing techniques generalize to novel imaging datasets with heterogeneity in acquisition parameters and clinical phenotypes.

Public Health Relevance

This research will make it possible for a wide community of researchers who collect and analyze medical imaging data to take advantage of a new class of computer algorithms that very accurately label and measure anatomical structures and pathological formations in medical images. By offering more accurate image-derived measurements, the project promises to improve the accuracy of diagnosis, reduce the costs of biomedical re- search studies and pharmaceutical trials, and accelerate scientific discovery.

National Institute of Health (NIH)
Research Project (R01)
Project #
Application #
Study Section
Biodata Management and Analysis Study Section (BDMA)
Program Officer
Pai, Vinay Manjunath
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
University of Pennsylvania
Schools of Medicine
United States
Zip Code