This project is further refining scientific and quantitative reasoning assessment tools and procedures for use in undergraduate programs. Although it is desirable that institutions of higher education begin to use direct measures of student learning to assess important learning outcomes, direct measures are currently the least systematically used of available assessment techniques. Without appropriate assessment methods, the nation will continue to rely upon less desirable indicators such as: student self-reports, actuarial reports, and external ratings of institutional quality. By exploring the generalizability of current instruments designed to measure scientific and quantitative reasoning skills at James Madison University to other diverse institutions, this project is contributing to knowledge of undergraduate STEM education, developing faculty expertise in assessment practice, and helping to build an interdisciplinary community of scholars from five diverse institutions. James Madison University (JMU) is uniquely qualified to contribute to the development and dissemination of psychometrically sound instruments and assessment practice due to its long-term commitment to this work via the Center for Assessment and Research Studies (CARS) (www.jmu.edu/assessment/). This project is building on successful work conducted over several years by CARS faculty with significant collaboration by JMU STEM faculty members through which objectives for scientific and quantitative reasoning have been carefully crafted, and innovative items have been created and mapped to these objectives. JMU is currently using the eighth generation of instruments designed to measure collegiate scientific (SR) and quantitative reasoning (QR) skills and knowledge. This project is building on our existing research base that has demonstrated the reliability and validity of scores. Recent research supports the hypothesis that current scientific and quantitative reasoning goals and associated assessment instruments can successfully be modified for use other institutions in need of sound assessment methods and practices. The project is working on the following six objectives: 1. Exploring the psychometric quality and generalizability of the SR and QR instruments to institutions having diverse missions and serving diverse populations. 2. Building improved and scientifically based assessment plans for adoption at home institutions through consultation and participation in Faculty Institutes. 3. Building assessment capacity at participating institutions through professional development in assessment practice, analytic methods, and data presentation to enhance curricular reflection and improvement. 4. Developing new assessment models and designs for adoption or adaptation by other institutions. 5. Documenting potential barriers to effective assessment practice and exploring solutions. 6. Creating scholarly communities of assessment practitioners to sustain work at participating institutions and beyond. Intellectual Merit: This project is advancing knowledge and understanding of student learning in scientific and quantitative reasoning skills and providing sound tools for other projects and researchers. The project is following the National Research Council (2001) assessment model through formation of interdisciplinary teams from each institution with broad science, mathematics, cognitive psychology, and measurement expertise to study the generalizability of assessment tools to distinct institutions serving diverse populations. Obstacles to effective practice at each institution are being identified and addressed. Broader Impacts: This project is improving our understanding of student scientific and quantitative reasoning in order to enhance student learning and teaching. The project is directly providing advanced training opportunities for faculty and administrators from diverse participating institutions. These interdisciplinary teams represent a new community of scholars that are contributing to research on STEM teaching, learning, and assessment at their home institutions and beyond via rich dissemination opportunities.
