Averaging about $1,000 per patient1, recruitment remains an expensive bottleneck for human studies. The rapidly increasing adoption of electronic health records (EHR) has made electronic prescreening (E-screening hereafter) a practicable solution to this bottleneck. Our long-term goal is to achieve this "holy grail". Our short- term goal of this competing continuation is to develop an intelligent patient query consultant to improve the accuracy and efficiency of E-screening. One of the difficulties for E-screening is the semantic gap between eligibility criteria and clinical data.2 Each eligibility criterion (e.g., hypertension) describes a patient characteristic, which is correlated with multiple data features (e.g., orders of hypertension drugs, elevated blood pressure, and symptoms of hypertension) in EHR. Moreover, each data feature may have multiple semantic representations (e.g., "SBP", "BP", or "blood pressure") from disparate data sources. For example, elevated systolic blood pressure can be recorded in varying formats in an emergency room, a doctor's office, an ICU, and an in-patient unit, but not all of these readings necessarily indicate chronic hypertension. The use of clinical data to identify patients eligible for clinical research requires specialized knowledge and expert guidance to navigate the vast space of data features and intelligent inferences from data features for eligibility determination. A user must understand the characteristics of available data before using them to search for patients. For example, when only 5% of hypertensive patients have ICD-9 codes for hypertension but 73% of these patients have hypertension drug orders, using drug information to construct a query of hypertensive patients will be more effective than one using ICD-9 codes. Even sophisticated biomedical data query tools such as i2b2, VISAGE, and STRIDE only passively translate user-specified data features into a query statement. They do not guide a researcher in selecting a data feature and its most appropriate semantic representations or data sources. Little aid is available to inform researchers about data characteristics or to help them conduct exploratory data analyses for optimal data feature selection. Mixed-initiative interaction,3 which allows human and computer to collaboratively contribute to converged problem solutions, can potentially fulfill this need. We hypothesize that by equipping biomedical researchers with a knowledge-based, mixed-initiative dialog system, we can maximize the efficiency and accuracy of E- screening by supporting exploratory analyses of correlated data features for query optimization. Our approach is innovative because it (1) addresses the user needs for intelligent query interfaces for clinical data, (2) provides a novel data-driven approach to eligibility determination based on correlated data features, and (3) enables efficient query optimization through support of human-computer collaborative problem solving. We will build on the results from our first funding period for bridging the semantic gap.4-21 We developed an analysis pipeline called EliXR to construct a semantic knowledge representation for eligibility criteria 6,9,16,17, which can be used to transform free-text eligibility criteria ito structured narrative.6 We developed methods to dynamically categorize eligibility criteria by data type.8 We accumulated E-screening experience from three NIH-sponsored clinical trials.7,13,21 We developed a method combining PubMed knowledge and EHR data to infer patient phenotype4 and reconciled structured and unstructured clinical data to support E-screening.18 We are prepared with methods and a preliminary understanding of the building blocks necessary to optimally translate eligibility criteria into data features;therefore, our current proposal is the logical next step.
Our specific aims are to: 1. Use mixed methods to understand the needs of biomedical researchers for query clarification and identify common strategies used by query analysts for plan optimization for complex eligibility queries. 2. Develop a knowledge-based, mixed-initiative dialog system to improve human-computer collaboration for query formulation using participatory design methods. 3. Evaluate the efficacy and usability of the mixed-initiative dialog system using a research data warehouse and two use cases: research protocol feasibility testing and trial recruitment prescreening. We will advance the field by contributing knowledge of the needs for query support among biomedical researchers and an effective E-screening method that combines intelligent query recommendation and iterative query by review22 to improve data access for researchers through human-computer collaboration.

Public Health Relevance

This project addresses the needs of clinician scientists to understand data characteristics when using clinical data to search for potentially eligible clinical research participants. It will develop a novel intelligent query consultant to guide clinicians in interrogating clinical databases. This project has the potential to improve the efficiency and accuracy and reduce cost of patient prescreening for clinical and translational research.

National Institute of Health (NIH)
National Library of Medicine (NLM)
Research Project (R01)
Project #
Application #
Study Section
Special Emphasis Panel (ZLM1-ZH-C (01))
Program Officer
Sim, Hua-Chuan
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Columbia University (N.Y.)
Internal Medicine/Medicine
Schools of Medicine
New York
United States
Zip Code
Hruby, Gregory W; Hoxha, Julia; Ravichandran, Praveen Chandar et al. (2016) A data-driven concept schema for defining clinical research data needs. Int J Med Inform 91:1-9
He, Zhe; Ryan, Patrick; Hoxha, Julia et al. (2016) Multivariate analysis of the population representativeness of related clinical studies. J Biomed Inform 60:66-76
Hoxha, Julia; Jiang, Guoqian; Weng, Chunhua (2016) Automated learning of domain taxonomies from text using background knowledge. J Biomed Inform 63:295-306
Hoxha, Julia; Chandar, Praveen; He, Zhe et al. (2016) DREAM: Classification scheme for dialog acts in clinical research query mediation. J Biomed Inform 59:89-101
Gordon, Claire L; Weng, Chunhua (2016) Bacterial clinical infectious diseases ontology (BCIDO) dataset. Data Brief 8:881-4
Ma, Handong; Weng, Chunhua (2016) Prediction of black box warning by mining patterns of Convergent Focus Shift in clinical trial study populations using linked public data. J Biomed Inform 60:132-44
Hruby, Gregory W; Matsoukas, Konstantina; Cimino, James J et al. (2016) Facilitating biomedical researchers' interrogation of electronic health record data: Ideas from outside of biomedical informatics. J Biomed Inform 60:376-84
Hruby, Gregory W; Rasmussen, Luke V; Hanauer, David et al. (2016) A multi-site cognitive task analysis for biomedical query mediation. Int J Med Inform 93:74-84
Sen, Anando; Ryan, Patrick B; Goldstein, Andrew et al. (2016) Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials. Ann N Y Acad Sci :

Showing the most recent 10 out of 75 publications