Background: One-third of all U.S. physician residents rotate through a VA medical center and are involved in 37% of VA's physician services while caring for 2.5 million veterans. Despite the critical role that VA professional staff plays in supervising trainees in patient care, no scientifically valid measure exists to quantify the intensity of that supervision. Thus, little is understood as to how supervision may impact patient outcomes, guide trainees'progress towards competencies to enter independent practice, or contribute to producing clinical workload to meet the health care needs of veterans. Purpose: The purpose of this study is to assess the performance of the Clinical Supervision Index (CSI), a VA conceived instrument designed and developed by the investigators through an HSR&D pilot study and VHA/OI&T Greenfield award to quantitatively measure the intensity of the supervision that trainees receive during inpatient or outpatient encounters with pre-selected """"""""active"""""""" patients. Fully functioning at the VA Loma Linda medical center since May 2013 on class II software in CPRS, CSI's key features are: the measure is applicable across all health professions education trainees (physicians, nurses, psychology, pharmacy, etc.) and over inpatient and outpatient care settings and measures supervision for the clinical care described in trainee-signed progress notes. We maintain a registry of trainees and supervisors within the medical center. In addition, our CSI software automatically administers electronic Trainee and Supervisor Surveys through CPRS, searches across VistA for the responsible supervising practitioner, creates merged files on regional servers, and electronically transfers those files to the Allocation Resource Center for processing. Objective: We assess CSI concurrent validity (do the lengths of supervised time that trainees and supervisors report in CSI surveys agree with time computed from direct observations?);construct validity (are supervision levels predictable?);and applicability (do adjusted CSI intensity scores reflect trainee's clinical competencies and predict patient care outcomes, patient satisfaction with trainee- involved VA care, trainee education outcomes, and trainee satisfaction with their learning experiences?) Method: Data are obtained at the VA Loma Linda medical center from CSI electronic files, VistA extracts (supplemented with CDW data), face-to-face quarterly patient surveys during one-year patient follow-ups, web-based semi-annual trainee satisfaction surveys during the two-year trainee follow-ups, panel of 40 trainees, panel of 300 patients with type II diabetes enrolled in the continuity clinic, 90 supervisors, involving 1,500 encounters (indexed progress notes), with 750 encounters sampled for direct observation. The CSI has been fully operational at the study site since May 2013. Analyses include intraclass correlations, mean response biases, and effect sizes computed from Generalized Estimating Equations to account for hierarchical trainee, supervisor, and patient encounter level data.
To ensure quality care for its veterans and education opportunities to future health professionals, VA relies on its medical staff to supervise students, interns, and residents from affiliated education institutions who participate in patient care at VA medical centers. While entrusted with the care of our Veterans, there are no scientifically validated instruments to quantitatively assess the intensity by which trainees are supervised in clinical teaching settings. This study will assess the performance of an electronic, fully automated, Clinical Supervision Index developed by VA to permit VA leadership to understand how costly supervision impacts patient's health outcomes, trainee's professional progress, and trainee contributions to facility workload.
