This project is developing and field-testing instructional design concepts and educational software to teach diagnostic skills necessary to identify and solve problems in complex technical systems. Using concept mapping software, along with expert-system programs, the overall software package enables junior and senior-level technology and engineering students to benefit from personalized, iterative interactions that enable them to design and evaluate their analytical processes for the diagnosis of a problem, compare thesir processes to those of expert diagnosticians, and then refine their approaches. Advancing diagnostic skills training in the undergraduate technology and engineering curriculum is based on several theoretical anchor points: the first is the cognitive development theory associated with the higher-order thinking skills of analysis and problem-solving; the second is the application of hierarchical learning strategies in the acquisition of concrete, abstract and process concepts in technical systems; and the third is the proven practice of using iterative feedback, rubrics and expert examples in improving learning. These foundational theories and principles are being integrated into educational software developed and delivered by stand-alone or networked computers, thereby adding to the toolkit of STEM instructors. Overall, the strategy to increase the level of advanced technical diagnostic skills is providing industry, government and the military with a smarter workforce to improve our nation's overall competiveness. The outcomes of the project are also contributing to the general STEM education knowledge base.
The National Science Foundation awarded funding to Indiana State University, College of Technology faculty George Maughan, A. Mehran Shahhosseini and Tad Foster to develop a program to bolster training for students in a variety of science and technology fields. The grant entitled "Advancing Diagnostic Skills Training in the Undergraduate Technology and Engineering Curriculum" ran from February 15, 2012 to January 31, 2015. Overall, the purpose of the project was to experiment with the development of a self-contained training program to introduce engineering and technology undergraduates to conceptual mapping techniques for the purpose of enhancing their diagnostic skills for technical problems. The project staff developed a two-hour, computer-based training program using the Lectura authoring software. The first hour of this training was designed to introduce students to systems and troubleshooting problems. During this phase of the training, the subjects were trained to use the Visual Understanding Environment software to develop conceptual and process maps. Also during this phase, the subjects were given a "simple" technical problem and asked to map out a plan of action. Their map was then electronically compared using a similarity flooding algorithm to an expert’s map that was already encoded. The information developed by the algorithm was returned to the subjects as feedback so they would know how well they did. The second phase of the training took the form of two technical problems developed in collaboration with industrial partners. The first problem involved an electrical grid distribution system and the second problem focused on a malfunctioning heat exchanger system. In both cases, the subjects were provided with a description of the system and tables containing operational information. From the information presented, the subjects were asked to analyze the system and develop a process map for addressing the apparent malfunction. The evaluation of this project included pilot-testing the training program after the completion of the first hour, after the completion of the electrical grid problem, and after completion of the full training program. The evaluation also included a satisfaction survey of the finished product as well as experimental data collection. The experimental portion involved subjects from five universities and six majors in engineering or technology. The subjects were provided the program and all needed software on a memory stick and typically completed the training on a laptop. In short, the subjects during pilot testing and during experimentation found the training to be interesting (X? = 3.09 on a four-point scale), and useful (X? = 3.22 on a four-point scale). However, the subjects’ level of performance was much lower than expected on average. It was clear that about 20% of the subjects clearly understood and found the mapping technique and the feedback provided to help them improve their diagnostic skills. In general, this was not the case. That is, performance on the two main tasks was low and remained low regardless of the type of feedback given. A major contribution of this work was the development of automatic student feedback by programming the software using a similarity flooding algorithm. This algorithm provides a means of comparing two graphic items such as conceptual maps. The project team modified the original algorithm to include an automatic thesaurus for synonyms, the use of relative similarity versus full similarity, weighting the nodes, and setting threshold values. Each modification significantly improved the algorithm’s accuracy, so that it was possible to compare various expert maps with themselves and get a 100% matching indication. The team validated the algorithm using multiple tests. Additional information about the project can be found at www.diagnosticskills.org/index.htm or by contacting the project’s principle investigator, Dr. W. Tad Foster at tad.foster@indstate.edu (812-237-4508).
