There is a clear and accelerating trend toward making trainings available to alcohol and drug abuse treatment clinicians and clinical staff via the internet and distance learning methods (e.g., Weingardt, Cucciare, Bellotti, &Lai, 2009). This movement is cost effective and offers tremendous possibilities for dissemination of evidence-based practices to audiences for which access had been previously limited or unavailable. However, these benefits come with a cost, especially with regard to clinical skill evaluation. These dissemination efforts are moving faster than the development of tools to assess the clinical skills of trainees who participate in these technology-based training efforts. Role play is a critical method by which clinicians can practice a clinical intervention, receive feedback, and monitor application of that feedback to their clinical practice (e.g., Bosse et al., 2010). Remote and asynchronous training pose significant challenges when conducting role plays or similar unrehearsed assessments of clinical skill. While some methods are available, such as video conferencing, these can often rely on expensive equipment and technology, and may be impractical for most field clinicians to use. This project proposes to build on existing data collected as a part of a previously funded R01 award that examined different kinds of post-training support for alcohol and drug abuse clinicians (Barrick &Homish, 2011). Using a Generalizability Theory-based approach (e.g., McGillicuddy, Rychtarik, &Morsheimer, 2004;Cronbach, Gleser, Nanda, &Rajaratnam, 1972) we will develop an alternate form of the telephone-based clinical skill assessment tool (T-CAT) and conduct a psychometric evaluation to examine the generalizability, alternate form reliability, and construct validity of the T-CAT. This method employs a telephone-based approach using interactive voice recording (IVR). Clinicians call a toll-free telephone number, listen to a series of clinical scenarios, and speak their responses to the situations. The responses are recorded using IVR, then coded for clinical fidelity and skill, and rated by the research team. This approach has promise as a method to allow for clinical skill assessment for distance learning and web-based trainings in the alcohol and drug abuse treatment fields.

Public Health Relevance

The societal and personal consequences associated with alcohol and drug misuse are profound. Despite ongoing advances in addictions, a significant gap remains between research and practice in the substance abuse treatment field. This project aims to develop a new method of assessing clinical skill of community clinicians after a training workshop. The implications of the proposed study are significant in that there could be a new approach to assessing the implementation of empirically supported evidence-based practices, such as Behavioral Couples Therapy (BCT). This tool could help ensure that not only knowledge, but clinical skills are transferred successfully into practice.

National Institute of Health (NIH)
National Institute on Alcohol Abuse and Alcoholism (NIAAA)
Exploratory/Developmental Grants (R21)
Project #
Application #
Study Section
Dissemination and Implementation Research in Health Study Section (DIRH)
Program Officer
Lowman, Cherry
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
State University of New York at Buffalo
Organized Research Units
United States
Zip Code