This collaborative proof-of-concept study involves The University of California at Santa Cruz, The University of Texas at Austin, and the Los Alamos National Laboratory. The PIs will test the efficacy of using agent-based simulation and visualization models to identify the factors that predict mathematics achievement for students from the 8th grade to the 12th grade and beyond. The team are using a data set that includes 14 years of data on student grade reports, coursework, demographics, teacher variables such as years of service, professional development courses take, years of service, and other artifacts. The investigators hypothesize that agent-based modeling can be used to improve mathematics education. The research questions is What are the predictors of success in mathematics of public school 8th grade students and beyond as measured by a) mathematics performance (test scores) broken down by different mathematical skills? b) enrollment in algebra class (8th grade and high school)? and c) algebra and mathematics grades in 8th grade and high school? This exploratory study will analyze data using three tasks. The first task involves data assessment. The first task will involve discovering distributional information in general. They will explore visual and analytical processes of different variables so that different synthetic data can be simulated. The second task involves collaborating with a statistical science team to incorporate distributional information so that multivariate samples can be generated to form synthetic populations to use to build the agent-based model. The third task involves using the actual data from two large school districts to understand and quantify variability in the data.
Education systems do not have a valid way to measure progressions of mathematics education to evaluate outcomes associated with mathematics learning outcomes. This project will provide a baseline understanding of student's progression in mathematics achievement that is critical in helping educators and policy makers set goals and standards for mathematics education within the United States.
The primary outcome of this work is a web-based visualization tool offering several perspectives on the data generated by the agent-based model and on its evolution. It also uses the some of the same visualization techniques to present data from the population on which the model is based. The tool focuses on presenting aggregate data at discrete points in time (that is, school years) as well as on tracking students though time. Histogram plots allow users to visualize 12 years of student grades generated by the agent-based model. Parallel coordinates are used to visualize combinations of model inputs – student abilities, grades, teacher abilities, and aggregate classroom abilities – across school years. This visualization may be used to isolate particular groups of students to better assess the impact of various model parameters. Finally, alluvial coordinates track the aggregate flow of students between different performance tiers, giving an overall picture of student performance trajectories. Each of these visualizations presents outputs from multiple iterations of the agent-based model, illustrating the evolution of the model across many cycles of calibration. The aggregate histrogram visualizations are also used to display real student mathematics grades. All of these visualization techniques are meant to provide stakeholders alternative and novel methods of looking at student data and, in particular, considering how various factors may contribute to student performance. The web-based application also includes an animation illustrating how the agent-based model works – that is, how students (agents) are assigned each year to classrooms and their grades are calculated as a function of their ability, prior mathematics grade, teacher ability, and average classroom ability. Finally, the tool is accompanied by a narrated walk-through explaining its features and demonstrating its operation.