This SBIR Phase I project proposes to develop a novel diagnostic algorithm and methodology for obtaining detailed learner feedback in multi-step problem solving questions. This feedback is fundamental to derive learning patterns, identify at-risk learners, assess knowledge mastery, and inform both students and teachers about what was learned, what needs to be learned, and where and how mistakes were made. It informs the teacher what the students know and don?t know. The wealth of data collected by the proposed prototype will promote peer standards of excellence and student self-regulated learning, inform teaching practice, revise curricula, and inform educational administrators. This project is responding to a need in the educational system for personalized learning experiences for every learner, evidence-centered learning, and data that will improve teaching efficiency and effectiveness. The diagnostic assessment of the product will help educators understand the extent to which learners have mastered critical Science, Technology, Engineering, and Mathematics skills to identify specific areas in which improvement is needed and in which areas learners have mastered.
The broader/commercial impact of this proposal is the real-time student feedback and the subsequent analysis of that data to efficiently transform Science, Technology, Engineering, and Mathematics education. It provides a formative, low cost technology product that can help teachers personalize and revise curriculum in real-time and promote step-by-step problem solving skills required for 21st century careers. This will help the U.S compete effectively in the global market. The in-depth learner database informs school administrators and education policymakers on changes required to improve student performances and teaching practice, and compliance with recently adopted common core state standards. Data mining across multiple levels - from single problem to school to district to state and internationally - allows school administrators to compare performances within the U.S. and abroad. This project allows for the discovery of knowledge gaps while enhancing teaching, learning and training in all educational settings.
In January 2014, the National Science Foundation awarded YourLabs, LLC, an e-learning startup, a Phase I SBIR grant to create an innovative Knowledge Evaluation System (KES). The goal of KES is to improve teaching practice and students learning in Science, Technology, Engineering and Mathematics (STEM). YourLabs, LLC created KES, deployed it at www.yourlabs.com and two independent education experts conducted an efficacy study. KES is providing a solution to support efficient and effective instructions in and out of the classroom and identify at-risk students. It is a diagnostic application to assess students’ knowledge mastery in real time. It is not a test-taking software. Teachers select one or more STEM-based multimedia problem solving questions from YourLabs’ community library and assign them to students in or out of class. After the students answer the question on any mobile or desktop device, KES automatically grades it and then displays the concept behind the question, the real-life relevance of the question, links to remedial material, and the step-by-step solution. The students respond to KES preset queries (response widgets) on their understanding of the concept(s) and each solution step, the type(s) of mistake made in a step, if any (e.g. sign error or used the wrong equation) and whether they need help. KES then analyzes the students’ responses and provides actionable analytics to the teacher and students. KES analytics goes beyond statistics on right and wrong answers, averages and standard deviations. In addition to these statistics, KES provides statistics on the amount of time students took to solve the question, the various answers submitted to a question, the understanding of the concept behind the problem, the understanding of the steps, the types of mistakes made at a given step and identifies students who need help. By pinpointing exactly where students are having the most difficulty and the type of difficulty, a teacher can take action to timely address any learning deficiencies. Students also view the analytics after completing the question and providing feedback. By comparing his/her performance with his/her peers, and studying the solution steps, a student can self-reflect on his/her knowledge mastery and take corrective actions. The independent efficacy study conducted by two education researchers from the University of Arizona’s College of Education concluded that YourLabs’ KES has a high potential in improving teaching practices and student learning. Students’ comments on KES include "It was an excellent conduit for learning and the advancement of my education."; "It will go through each answer and give a feedback and explain it, not just give a general idea."; "Gives me an idea of how well/bad the class is doing [analytics]."; "Not only were we able to work through the problems with enough time to complete each step, the solutions were readily available for review." YourLabs’ KES is gathering big educational data that can drive not only day to day instruction but also policy decisions. Such data includes performance from a problem level aggregated to state and international levels. Mining this data provides opportunities for research and teaching in STEM, improvements in the performance, skills and attitudes of members of underrepresented groups in STEM, and comparisons of performance among different schools within a district or state or nationally; where and what difficulties students have in these schools to drive policies, practices, training or other programs to effect improved outcomes. The decline of U.S. students’ performances in global competition in STEM subjects limits their career choices and U.S, competiveness. YourLabs’ KES is an innovative product that has broad applications beyond assessing competency and STEM skills in K-12 schools, postsecondary institutions and workforce training. The real-time student feedback collected by KES and the subsequent data mining and analysis are vital to efficiently transform STEM education. It helps teachers personalize and revise curriculum in real-time and promotes concept understanding, critical thinking and step-by step problem solving skills required for 21st century careers. It will also help the U.S. compete effectively in the global market. KES is making use of low cost technology to put in-depth learner data into the hands of students, teachers, parents, school administrators and education policymakers anytime, anyplace, and on any device. It is aligned with emerging standards such as Common Core State Standards and national programs such as Race to the Top, Bring Your Own Device, ConnectED, and, No Child Left Behind.
