Creating simulation-based performance assessment tools for practicing physicians A team of leading simulation researchers and educators propose a 10-center study of the applicability of mannequin-based simulation to high-stakes assessment of practicing clinicians who are seeking board recertification. This unique study will address important issues for performance evaluation of practicing clinicians, which include the development of standardized scenarios with associated reliable scoring instruments, distributed delivery and assessment of clinicians'performance using those scenarios across multiple testing sites, and a description of the distribution of medical/technical and behavioral performance of 200 practicing board- certified anesthesiologists (BCA). The results will inform competency assessment of physicians in other specialties as well as physicians-in-training. Using a network of experienced simulation instructors and programs already offering all-day simulation courses that are mandatory for BCA desiring Maintenance of Certification in Anesthesia (MOCA) provides a singular opportunity to address some of the challenges of measuring clinical competency using high-fidelity simulation.
The Specific Aims are to: 1) Develop standardized, generalizable simulation scenarios with associated valid, reliable performance assessment tools to conduct simulation-based assessment of BCA;2) Demonstrate that simulation-based clinical assessment can be reliably delivered across multiple national sites for the purpose of evaluating board-certified physicians seeking recertification;3) Describe quantitatively and qualitatively the distribution of clinical performance during simulation from a large and diverse cross-section of board-certified physicians;and 4) Begin to address the remaining challenges and questions related to reliable and valid simulation-based assessment of practicing physicians'clinical competency. The products of Aim 1 will be 4 standardized scenarios, associated reliable performance rating instruments, and detailed methods for conducting clinical performance ratings.
In Aim 2, we will conduct more than 100 MOCA(R) courses over a two-year period in 10 ASA-Endorsed centers;the clinical performance of 200 BCA will be assessed. We hypothesize that there will be a broad distribution of medical/technical and (especially) behavioral performance ratings from poor to excellent, reflecting a diverse cross-section of participants. Finally, we will hold a national stakeholders consensus conference to discuss the remaining challenges and questions related to reliable and valid simulation-based assessment of clinical competency. The results will delineate the requirements of future validation studies of clinical performance assessment and provide important guidance for simulation-based competency assessment across multiple specialties.
Assurance of physicians'ability to detect and manage uncommon but potentially lethal events is an area of particular societal concern and may be amenable to simulation-based performance assessment. This multicenter study will develop rigorous methods and measures for assessing the acute care competency of practicing physicians using mannequin-based high-fidelity simulation.
| Weinger, Matthew B; Banerjee, Arna; Burden, Amanda R et al. (2017) Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists. Anesthesiology 127:475-489 |
| Weinger, Matthew B; Burden, Amanda R; Steadman, Randolph H et al. (2014) This is not a test!: Misconceptions surrounding the maintenance of certification in anesthesiology simulation course. Anesthesiology 121:655-9 |
| Weinger, Matthew B; Gaba, David M (2014) Human factors engineering in patient safety. Anesthesiology 120:801-6 |
| McIvor, William; Burden, Amanda; Weinger, Matthew B et al. (2012) Simulation for maintenance of certification in anesthesiology: the first two years. J Contin Educ Health Prof 32:236-42 |
