This project is an extension of a previous effort to develop a clinically-based method for assessing quality of care delivered to pre-menopausal women, children, and adolescents enrolled in managed health plans. The proposed study will broaden this effort to include men under the age of 50 and men and women aged 50 and older. In the first phase, the investigators will develop evidence-based criteria for evaluating care in 15-20 conditions chosen for their importance as leading causes of morbidity and mortality within the target population. For each condition, process measures for care will be created based on the best scientific evidence available. An expert panel (consisting of generalists, specialists, and academic and community practitioners in both managed care and traditional settings) will be used to rate the suitability of the conditions and the criteria developed. In addition, these panelists will develop rules for weighting indicators to reflect potential health benefit or harm to individuals as a result of complying or failing to comply with the indicators. These process criteria will be translated into performance measures linked to the care process. After the indicators are developed, the investigators will design a computer-based data collection tool consisting of multiple modules containing indicators related to each of the target conditions. Branching logic will be employed to dictate entry into a particular module. Next, they will design a sampling strategy that uses both enrollment and administrative data to identify eligible enrollees and will conduct a pilot test in order to check both the sampling plan and data collection tool. Specifically, they will test the reliability and validity of administrative data entered into the data collection tool against medical record data obtained by chart abstraction. Additionally, this pilot will test the performance of the data collection tool in hands other than those of the investigators. In the second phase of the project, evaluation sites will be selected based on geographic and organizational diversity as well as availability and reliability of administrative data. Proposed sites include United HealthCare, PacifiCare, Harvard Pilgrim Health Care, and U.S. HealthCare. Programmers at each site will implement the sampling strategy to identify a random sample of 500 enrollees at each of the four sites, and registered nurses at sites will be trained in the use of the data collection tool. A nurse based at RAND will be responsible for oversight and reliability testing for the data collection process. The administrative data will be checked against the medical record. Error rate will be noted by type of diagnosis and service. Then, quality of care scores will be created that aggregate indicators by condition, acute vs chronic care, site, and care function. The score will be the number of processes done correctly divided by the number of processes for which each enrollee was eligible, or an overall average adherence proportion. A subset of the quality scores will be constructed using both administrative and medical record data to determine if administrative-based scores differ from medical record-based scores. In addition, sensitivity of scores to different weights assigned to reflect the consequence of receiving no care or substandard care on patient outcomes will be assessed along, with the adequacy of severity adjustment. Resource costs will be tracked to determine the costs a health plan would expend in conducting this global quality evaluation, and finally, a sensitivity analysis will be performed to predict the standard error in quality scores for each function category at each site assuming available information from a decreasing number of medical records. This will allow an estimate of tradeoffs between costs and precision of quality estimates. These methodological analyses will be used to establish a fair scoring method that can be used to explore questions related to variation in quality among health plans. First, the investigators will use repeated measures of analysis of variance and regression to test whether there are differences in health plans on quality. Primary dependent variables will be the quality scores developed for each function category (e.g., screening and prevention, diagnosis, treatment, and follow-up). Next, they will explore variation by function within as compared to between plans and finally, they will pool data to explore whether enrollees with particular characteristics are at increased risk for receiving low quality care. The analysis will attempt to create summary measures was well as condition-specific ratings. The final step in the project will be documentation and dissemination of the final data collection tool in the form of a public domain software package that is easily transportable to common laptop hardware platforms. This package, to be made available through the National Technical Information Service, will include documentation on how to use the tool, as well as descriptions of the final indicator set and recommended sampling and analysis techniques.
