This research project will develop psychometric methodology to empirically evaluate developmental progressions. A developmental progression describes a theorized or observed sequence of cognitive, psychological, or behavioral developments in an individual or group. In educational settings, learning progressions describe the increasingly sophisticated ways of reasoning that develop as students learn about specific content domains over time. Despite their prevalence and utility, quantitative methodological developments to evaluate learning progressions have stagnated. This project will advance the fields of psychometrics and learning sciences by providing a modern, multidimensional, and longitudinal framework for modeling developmental progressions. Although the project focuses on educational applications, the developed methods will be widely applicable in disciplines across the social and behavioral sciences. The project will train a graduate student from an underrepresented group. Free and easy-to-use software will be developed for researchers to utilize in their own examinations of learning progressions. The results and products stemming from this project have the potential to change the way researchers design, interpret, and analyze assessments in the empirical evaluation of developmental progressions.
The investigator will use a diagnostic classification model (DCM) framework to model learning progressions. DCMs are multivariate psychometric models that classify examinees into specified levels of categorical latent traits (e.g., basic, proficient, advanced). DCMs have become attractive in educational settings because they provide much desired diagnostic and criterion-referenced score interpretations in the form of classifications. Recently, DCMs have been developed for longitudinal contexts that provide criterion-referenced interpretations of student growth. To model learning progressions, the developed model will combine a generalized longitudinal DCM with the hierarchical DCM designed to model attribute hierarchies. This fusion of modeling frameworks allows for the simultaneous examination of attribute hierarchies and student learning over time, which together comprise the basis of a learning progression. Simulation studies will guide and inform the practical application of the developed methods with respect to data requirements (i.e., number of items or sample size), test design, model fit, and factors impacting the accuracy, validity, and reliability of model-based inferences.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
