Comprehensive and standardized biopsychosocial assessments are a key piece of infrastructure in the drive towards evidence-based and effective substance abuse treatment. Demands to cover more areas and constructs mean that such assessments can take 1-2 hours - which is a lot relative to the 4 hours of treatment typically received in the public treatment system. Computerized adaptive testing (CAT) is a promising approach for improving assessment efficiency without compromising assessment validity or reliability, but faces several challenges when applied to clinical assessments. These challenges include using CAT to measure multiple domains or dimensions, assist in treatment planning by triaging persons into low, moderate and high symptom severity, detecting persons with atypical presentation of symptoms, and detecting and controlling differential item and test functioning as it relates to various patient characteristics (e.g., age, gender, race, primary substance). The proposed project seeks to develop a coherent methodology for creating CATs for standardized biopsychosocial assessments that is applicable to diverse populations and effectively addresses the above issues. A series of simulations using existing data are proposed to compare several potential CAT methods (and combinations of methods) in terms of their ability to improve the validity and efficiency of clinical assessment (the same information with fewer items). The existing data and scales come from a standardized biopsychosocial called the Global Appraisal of Individual Needs (GAIN). This has the properties required to address the aims, is widely used (over 702 sites in 43 states, 4 Canadian provinces, Mexico and England), with a large data set that is already pooled (n=18,639 adults and adolescents entering a subset of 132 treatment sites as of 12/31/2007), that is diverse in clinical problems, age, gender, race and has 6- to 48-month follow-up interview data with an 80% or higher follow-up rate of those interviews planned for a given site/wave.
The specific aims of the proposed study are to compare multiple methods in order to improve the relative efficiency of CAT without losing reliability or construct validity in three ways: (1) Simultaneously estimate problem severity in a set of subscales with a common second order factor scale;(2) Vary precision around triage cut points designed to guide clinical decision making and be the focus of precision;and (3) Identify persons with atypical clinical profiles that impact clinical decision making. We will also assess the extent to which the above methods impact or need to incorporate differential item functioning by type of substance disorder, age, gender, race, or by time. The work in this initial proposal is all based on secondary analysis and simulations of CAT. However, the results of this work will then be used to prospectively evaluate the final CAT in a subsequent application.
While demands for more comprehensive assessment are key to moving the substance abuse treatment field towards evidenced based practice, this can take 1-2 hours and be disproportionate to the 4 hours of treatment typically received. Computerized adaptive testing (CAT) has the potential to reduce this burden on clients and programs while still providing reliable and valid information to support clinical decision making. This effort will compare multiple CAT methods to see which works better (alone or in combination) to address some of the key challenges in clinical assessment.
| Conrad, Kendon J; Riley, Barth B; Conrad, Karen M et al. (2010) Validation of the Crime and Violence Scale (CVS) against the Rasch measurement model including differences by gender, race, and age. Eval Rev 34:83-115 |
