The goal of this proposal is to conduct a thorough review of potential metrics that could be used in any study using eHealth interventions and create a standardized array of recommended metrics that will support eHealth studies. The VA has made a substantial investment in eHealth technologies to improve care for veterans. However, VA's ability to gauge impacts hinges on the measurement tools that they have available and ultimately choose to use. The metrics identified in this project will be critical to initiatives across a number of operations offices, including the My HealtheVet Program Office, Web and Mobile Solutions, the Connected Health Office, the broader Office of Informatics and Analytics, and the Office of Telehealth Services. The results of this RRP will provide critical insights about existing eHealth measures and identify gaps where new measures are needed.
The specific aims are: 1) to identify and evaluate existing metrics relevant to evaluation of eHealth implementation; 2) create a database of measures accessible to VA investigators and operations partners who are evaluating eHealth; and 3) complete a white paper to critically review existing measurement in the arena of eHealth for wide dissemination within the eHealth community. The study design will include a systematic search and review of multiple sources for currently available metrics that focus on patient-facing eHealth technologies. The process will include a systematic review of the literature, and identification and review of currently funded eHealth projects both within and outside the VA. Key concepts from existing models and published studies of technology use and adoption will be used to develop a working conceptual model to focus the review. Search results will be transferred to a reference database; titles and abstracts will be reviewed by investigators to exclude those that do not fit the definition of a metric. Inclusion criteria will be broad in order to assess the full extent of metrics available, and include: 1) the metric focuses on a patient eHealth-specific construct; and 2) the metric is available for review. The review will not be limited to metrics that report psychometric results of development or validation. The rationale for this is to obtain the most comprehensive list of what's out there and what could be developed further. An inventory will be developed which lists each metric along with empirically developed reliability or validity information. Reliability estimates will be explored using the range of current statistics available (internal consistency, test-retest reliability, alternate forms reliability) as will validity estimates (content validity, construct validity, predictive validity, discriminant validity). Each metric will be described using a uniform format, including a brief overview of the instrument's development, scoring procedures, psychometric properties, key references on the development and/or use of the instrument, and the actual scale (if available). Findings will be disseminated to the VA operations and research community through a cyber seminar and white paper. The compendium will be uploaded to the eHealth QUERI website and the Grid-Enabled Measures (GEM) database to facilitate use by practitioners and researchers.
The VA is investing significant resources into communication technology that allow veterans to electronically access information and communicate with their providers. This project proposes to assemble a set of metrics that can be used by practitioners and researchers to measure the effectiveness of implementing these technologies. Improved access to and knowledge of these metrics to evaluate implementation of patient technologies will help the VA improve Veterans' experiences with technologies and assess whether VA is making a difference in quality of care.
