We propose to develop the infrastructure for and deploy a commercial installation of an Algorithm Evaluation Service (AES). The service will help bridge the gap between algorithm researchers and commercial product developers. It will (1) assist commercial company in determining which medical image analysis and informatics algorithms they should integrate into their products, and (2) provide algorithm researchers with better access to clinically relevant amounts of data and with a better understanding of clinical and commercial needs. Specifically, the AES will provide a service whereby commercial organizations can post clinical data analysis challenges (data, metrics, and awards tied to performance milestones related to their intended products) and researchers can easily incorporate those challenges into their algorithm development workflows. Our successful Phase I grant culminated with our prototype system maturing and serving as the online infrastructure for the Multimodal Brain Tumor Segmentation (BRATS) Grand Challenge at MICCAI 2012 and the Prostate Segmentation Grand Challenge at ISBI 2013. Herein, we propose to (Aim 1) extend our system to support a novel mechanism for algorithm submission based on virtual machine technology that addresses clinical integration (i.e., multi-step data processing, including human interaction), security, and computational resource scalability to support extensive testing. We will (Aim 2) extend existing software development tools (i.e., our popular CMake build system) to make the submission of algorithms to AES challenges an inherent and effortless part of algorithm development for researchers. We will (Aim 3) validate the resulting system using additional grand challenges, and we will deliver it to and receive feedback from our first commercial customer as part of the proposed work. Specifically, two academic groups (Ohio State University and The University of Utah) have agreed to conduct grand challenges using our systems. Additionally, a commercial group (Imaging Endpoints) has agreed to serve as our first commercial customer. They are an imaging core lab that provides algorithmic solutions to pharmaceutical companies and clinical research organizations. They will use our AES to post a client's data and metrics, offer a prize, and thereby attract algorithm developers to generate solutions to their client's problem. It is generally accepted that a chasm exists between algorithm researchers, the capabilities of medical devices, and the needs of clinical practice. The proposed work will help bridge that chasm and will operate as a viable business model.
A chasm exists between researchers and medical device manufacturers. Commercial companies have a difficult time determining which medical data analysis algorithms they should integrate into their products because research publications often involve only limited clinical evaluations. Similarly, algorithm researchers are often frustrated by how little access they have to clinically relevant data and commercial needs. We propose to deploy an Algorithm Evaluation Service (AES) whereby commercial organizations can post clinical data analysis challenges (data, metrics, and awards tied to performance milestones related to their intended products) and researchers can easily incorporate those challenges into their algorithm development workflows. The needs of commercial companies and researchers will be met, and more effective commercial solutions to healthcare problems will result.
| Ding, Hao; Wang, Chao; Huang, Kun et al. (2015) GRAPHIE: graph based histology image explorer. BMC Bioinformatics 16 Suppl 11:S10 |
