State of the art diagnostic assessment procedures for externalizing disorders of childhood (i.e., ADHD and the disruptive behavior disorders) rely on reports from multiple informants (e.g., parents and teachers), often using multiple measures of the same construct, raising the challenge of how to interpret discrepant scores. Researchers and clinicians often use idiosyncratic decision rules for weighing the various reports, which is inconsistent with the recent attention and emphasis on evidence based assessment (EBA). By quantifying the degree of measurement consistency across types of raters, Generalizability Theory (GT) can provide an empirical basis for interpreting the results from multiple informants. Therefore, a GT study of the measurement of externalizing disorders can make a significant contribution toward the goal of EBA. The long- range goal is to maximize the measurement consistency of childhood externalizing disorders. The objective of this application is to quantify the degree of measurement error associated with parent and teacher ratings of childhood externalizing disorders using data for a community sample from the NICHD Study of Early Child Care and Youth Development (SECCYD) and for an archived clinical sample from The University of Alabama at Birmingham. These data are ideal, because they allow for multiple facets of measurement error that can be quantified by GT analyses. The first specific aim of this study is to partition the error variance in reports of externalizing symptoms across items, rater type, and grade (i.e., measurement occasion), as well as the interactions among these three facets. The second specific aim is to generate reliability estimates (Generalizability and Dependability coefficients) that incorporate all relevant sources of error. The third specific aim is to determine the generalizability of the various sources of information about these externalizing symptoms. All analyses will be conducted simultaneously for both ADHD and ODD behaviors in a multivariate analysis. Analyses will be replicated (for only one measurement occasion) with a clinical sample. The rationale for the study is that partitioning the error variance due to items, rater type, and grade will (a) determine which externalizing disorder (ADHD or ODD) are most reliably assessed by which informants, at which times, and thus (b) contribute to the further development of evidence based assessment.
Childhood externalizing disorders are linked to negative consequences and thus accurate assessment and diagnosis is critical. One of the major challenges of such an assessment is resolving how to interpret discrepant scores from different sources. The proposed study will help determine the best way to integrate assessment data from multiple sources for use in the diagnosis of externalizing disorders, which will contribute to the further development of evidence based assessment.
