Theoretical psychology has not, until recently, been in position to have much to say about how humans learn in dynamic, multidegree-of-freedom manual control domains. While applied concerns have led to successful training programs for manual control tasks (e.g., most people learn to steer automobiles in reasonable time), we currently lack the ability to predict how well people will do in these domains or how rapidly they will learn. This limits our ability to train people in these domains, where we presently rely on expensive one-on-one tutoring or similar intensive methods.
The project studies human performance and acquisition of sensorimotor tasks in real and virtual environments. Human motion data and performance of various skills by high performers and low performers, who exhibit linear performance gains, will be analyzed and compared to data for subjects who rapidly acquire skill and exhibit nonlinear performance gains. This data will inform the development of more accurate models of sensorimotor skill acquisition that can be expressed in ACT-R, and doing this should lead to improved understanding of training methods in human motor learning domains.
There are many domains of human performance that still require dynamic, multiple-degree-of-freedom manual control; this need arises in numerous human-robot interaction (HRI) tasks (e.g. piloting of unmanned vehicles, remote surgery, etc.). While applied concerns have led to successful training programs for manual control tasks (e.g., most people learn to steer automobiles in reasonable time), we currently lack the ability to predict how well people will do in these domains or how rapidly they will learn. Computational cognitive modeling has been highly successful in modeling and thus helping us understand learning and adaptation in a wide variety of domains. Cognitive architectures have a long history of supporting such models. However, such systems are still limited in their ability to model complex, dynamic manual control. In this research effort, we studied human performance and acquisition of motor skill in virtual environments, and developed computational models of human performance. Our work has revealed that overall performance measures often mask the underlying structure of the performance. Use of motion capture technology, gaming devices with motion sensors, and eye-tracking equipment informed the development of the computational models and drove our understanding of the movement strategies that produce high levels of performance. First, we compared sensor data of the Wiimote and computed acceleration data from a six camera motion capture system. The results show that the Wiimote’s performance is highly reliable and repeatable. For optimal performance, the acceleration data from the controllers needs to be filtered and gravity-compensated in post processing, as the data from the accelerometers of the controllers are noisy and sensitive to the direction of gravity. We used Neverball, an open source game in which the player controls the orientation of a platform on which a ball rolls, to study human motor control performance. In the game, there are coins (targets) on the platform which the player must collect to complete the level. The gaming controllers which control the platform orientation have onboard sensors that give us motion data related to the task performance. The motion based metrics of average frequency can be thought of as an implicit function of wrist movements while mean absolute jerk can be thought of as an implicit function of the smoothness of hand motions. These motion-based metrics governed the dynamics of the unconstrained complex dynamic task and the subjects varied the parameter values of these metrics to come up with various optimal solutions that would maximize the rewards or help in completing the level successfully. Computational modeling efforts have been spent attempting to get ACT-R, our modeling language, to learn to perform a simple dynamic target-hitting task the way the high-performing subjects do. The expert model’s performance is determined by several factors: (1) overall task strategy, (2) spatial parameters of the virtual targets generated by the model, and (3) noise in the motor movement system. We focused on the second issue, the size and location of the virtual targets. Altering the size of the virtual targets essentially moves the model along the speed-accuracy tradeoff curve. We ran a follow-up study where high-performing subjects were eye-tracked and more detailed analysis of their motion data were performed to gain insight into their task strategies. We believe that incorporating an updated model of movement endpoint distribution based results of those experiments into ACT-R will allow us to more closely mimic human subjects by adopting a more human-like strategy of faster movements with longer pauses between movements. The ACT-R motor system uses the simplest possible extension of the one-dimensional model into two dimensions: errors have the same (normal) distribution in all directions. Watching replays of expert subjects suggested that the off-axis error is, in fact, smaller than the on-axis error. We conducted follow-up experiments that indicate that the endpoint distribution in two-dimensional pointing is indeed not uniformly circular but instead is more of an oval, with a larger extent along the axis of movement than perpendicular to it. There are important broader impacts and practical applications of the work. Armed with a clearer understanding of the mechanisms underlying human manual control and its acquisition, we should be able to inform the design of HRI interfaces. The project created a unique interdisciplinary environment enabling education, training, and co-advising of graduate students, course development, and involvement of undergraduates in research. The PIs also participated in outreach activities on the Rice campus targeting underrepresented groups in science and engineering. The intellectual merit of the research stems from the analytical basis for modeling of human motor control, and the combination of control theory and human motor control to understand and leverage methods by which humans acquire manual control skills.
