This SBIR Phase I project proposes to determine the feasibility of new software implementations of formative assessment (FA) techniques for mathematics instruction. Tablet devices such as the iPad have an enormous potential to facilitate revolutionary change in education, but such potential is heavily dependent on the availability of appropriate software and services. The proposed activities will research the extent to which classroom FA techniques can be applied or enhanced when the teacher's role is augmented by a platform for computer aided instruction. They will seek to create adaptive learning environments that go beyond current state-of-the-art systems and that apply the adaptivity to free-form content delivered as instructional video segments and highly interactive problems, and to do so while maintaining a flow of content that feels natural. The proposed activities also have the potential to provide new tools for the analysis of effective techniques in mathematics education and interventions. The ability to model student understanding and analyze the learning process will lead to the creation of new learning analytics tools and enable additional research into effective practices for the teaching and learning of mathematics.
The potential broader impacts of the proposed activities include improved student access to mathematics; the increased participation of underrepresented groups in Science Technology Engineering and Mathematics (STEM); and improved STEM education through informed teacher practice and educator development. The proposed activities directly address mathematics education at a critical point in the curriculum: pre-algebra readiness, as set against the widely differing levels of mathematical fluency that emerge out of the elementary grades. Without a solid facility with pre-algebra mathematics, many subjects that are essential to participation in the modern world become difficult or impossible to master and academic confidence and commitment erode. More than any other subject, mathematical learning is cumulative, and as students fall behind their classmates, new material becomes less and less comprehensible and through this feedback they face an ever-widening gap to their peers. Formative assessment practices have been well-established as effective in closing these gaps and informing teacher decision making. It is by establishing toolkits with novel, software-based, formative assessment and helping to drive educator adoption of new formative assessment techniques that the proposed research may deliver these broader impacts. By addressing key customer needs, the project will create significant new commercial value within the educational market.
This SBIR Phase I project determined the feasibility of new software implementations of formative assessment techniques for mathematics instruction. Personalized learning through adaptive software has widely been identified as a critical enabling technology for education in the twenty-first century (see for example: The Council of Economic Advisors, 2011; Project Tomorrow, 2011). The topic of technology-personalized learning has a surprisingly long history, in fact stretching back to the early 1900’s. It is a history that has generated periods of great hype and bold projections, interspersed with letdowns and frequent controversy and critique (Benjamin, 1988; Meyer, 2014; Ferster, 2014). One addressable concern and criticism has been the lack of clear empirical data on the potential benefits of adaptive technology. In a 2013 review of educational technology, the Fraser Institute wrote that: Although informative in terms of how adaptive technology in education has evolved and developed, the review of research undertaken for this paper indicates a vast gap in sound, empirical research to determine and quantify the potential benefits from the adoption of such technology in education." (Izumi, Fathers, & Clemens, 2013, p. 7). In this SBIR Phase I project, we addressed this concern through the design and execution of a careful experimental study and the collection of empirical data on this type of adaptive technology in education (in our case, leveraging formative assessment techniques in the adaptivity). We conducted a multi-site randomized controlled trial that studied several forms of software-based adaptivity and demonstrated that Woot Math’s adaptivity contributed significant effects in learning and retention. In the trial, the only difference being tested was that the new adaptive capabilities (software implementations of formative assessment techniques) were enabled for the test group but disabled for the control group. The study results show statistically significant evidence that the adaptive version improved learning and retention, with moderate to large effect sizes in a sample of 350 students in grades 3-6. Full reports detailing the study have been published on the Woot Math website at https://wootmath.com/research. Our results establish a causal relationship – that the addition of the innovative adaptive capabilities resulted in improved learning outcomes. These results were also supported by a wide range of anecdotal evidence from student and teacher surveys and interviews indicating the overall effectiveness of the product. In addition to the randomized controlled trial, the project funding was used for the development of the Woot Math software application that was tested by both the control and experimental groups. In doing so, we created adaptive learning environments that went beyond current state-of-the-art systems and that applied the adaptivity to free-form content delivered as instructional video segments and highly interactive problems, all while maintaining a flow of content that feels natural. The student and teacher survey results indicate that the effort was successful in this way and delivered both high engagement and effective learning. The project also demonstrated the feasibility of new tools for analyzing the effectiveness of various practices for the teaching and learning of mathematics. The potential broader impacts from this project are improved student access to mathematics; the increased participation of underrepresented groups in Science Technology Engineering and Mathematics (STEM); and improved STEM education through informed teacher practice and educator development. Woot Math directly address mathematics education at a critical point in the curriculum: pre-algebra readiness, as set against the widely differing levels of mathematical fluency that emerge out of the elementary grades. Without a solid facility with pre-algebra mathematics, many subjects that are essential to participation in the modern world become difficult or impossible to master and academic confidence and commitment erode.
