This project will integrate new cognitive models of behavioral data based on queueing theory with new machine learning techniques for analyzing neurophysiological data, specifically electroencephalogram (EEG), in order to provide a deeper and more complete understanding of mental states as well as more accurate prediction of human performance. In cognitive modeling, a new brain network architecture for human performance and mental workload, called Queuing Network-Model Human Processor (QN-MHP), will be further improved. QN-MHP with a new human-like small-scale knowledge system will be used to model the increase of myelination in the brain in cognitive development and predict human performance, in terms of subjective risk perception and confidence. In machine learning, new spatio-temporal (pattern-based) classification techniques will be developed for multidimensional time series data and used to identify human mental states (e.g., fully awake, fatigue, distracted, anger) from EEG data. The integrated framework will result in a robust intelligent system that uses machine learning to identify mental states and the queueing model of that mental state to predict the human performance as well as provide a human operator with feedback. A mind-driven intelligent transportation system will be developed as a case study in this project, where a certain type of feedback will be designed to help drivers avoid accidents and to improve system safety. This system can also be applied to other human-machine systems that require full or partial attention of human operators (e.g., in aviation, military, or manufacturing settings).
We proposed a simulation model of human cognition system and pattern analysis of brain activity to study and predict human performance (such as numerical typing tasks, driving tasks, etc.). A new online brain activity monitoring framework that integrated both network model of cognition system and pattern recognition of brain data was developed as a result of this project. The framework was used to successfully identify changes in brain activity related to different cognitive states. The main application of our framework was to design of an innovative mind-driven intelligent transportation system with biofeedback. The outcome of this project provides profound impacts in both human performance modeling and brain activity analysis area. The undertaken work transforms a traditional simulation model to a rigorous mathematical model that integrates different aspects of human cognition and performance. In addition, it also bridges the gap between the theory of human cognition and the brain activity data from different cognition states. This project married these two domains and brought the strengths of each domain together in predicting important cognitive states of human (e.g., error states and distraction states). This outcome of this project also results in a new adaptive online time series prediction technique, whose efficacy was demonstrated through cognitive state prediction from brain electrical activity through electroencephalographic (EEG) signals. Significant findings of this project include (1) enhancement of cognitive model of human performance, (2) pattern recognition of cognitive sates from brain activity, (3) design of a mind-driven intelligent transportation systems with biofeedback. This project enhanced a new human-like computational model that contains detailed quantifications of human perception, cognition and motor control, knowledge of tasks and personality to improve the accuracy of human performance in a numerical typing experiment and a driving and navigation experiment. The new brain activity classification framework developed in this project was used to accurately identify cognitive states of human errors in a rapid perpetual motor task before an error occurs. In the driving and navigation experiment, our framework was also used to identify cognitive states of distraction and uncertainty in a navigation task before a distraction occurs. The project also proposed a new design of intelligent systems by integrating the capability of predicting drivers’ needs of route information. Particularly, an innovative navigation system could provide route information in advance based on a driver’s needs only when a driver is uncertain about route and intends to look at the map for assistance. The feedback from this system can prevent the driver from being distracted by reading a map passively. In turn, such a system could also provide reduction in providing feedback containing redundant, unnecessary navigation information to the driver as it often leads to drivers’ annoyance and takes up unnecessary cognitive resources from the primary driving task.
