The research objective of this award is to reverse-engineer human spatial control skills, with the goal of bridging the gap between human and autonomous systems. Adaptive and versatile spatial control capabilities are essential to successfully deploying unmanned aerial vehicles and other applications of autonomous control. understanding how humans achieve these skills is also important for the design of man-machine systems, such as active safety systems for helicopters or even tele-surgery systems. The research approach combines psycho-behavioral experiments and the application of control-theoretic principles. The working hypothesis is that human spatial behavior is determined by combining a type of model predictive control (MPC) process and a spatial value function (SVF). The MPC process describes how a trajectory is generated online,based on the immediate sensed environment; the SVF describes how the global environment and task elements are encoded and used in the MPC process. This model will then be used to study how the pilot adapts his/her strategies in the presence of disturbances, environment ncertainties, or when presented with novel ituations. Deliverables include the details of the odeling framework, documentation of research results, and outcomes of interdisciplinary student education and research experience.
If successful, this research will lead to developing new algorithms to model and replicate human daptability in spatial control tasks, paving the way for novel technologies for autonomous vehicle ontrol with unprecedented performance and adaptability. The improved understanding of human uidance skills will provide a gateway to new operator interfaces, augmenting human unctionality, effectiveness, and safety. Furthermore, the modeling framework will provide foundations for neurological studies, aimed at understanding the brain?s implementation of the patial control processes. Finally, the proposed interdisciplinary research activities will promote ew teaching and outreach activities, while attracting students from diverse backgrounds.
Summary of the Award Outcomes The overall goal of the research was to investigate the characteristics and principles of humans’ agile guidance capabilities using dynamical and optimal control principles. The research was conducted using interception tasks with remote-controlled miniature helicopters. The experimental facility uses motion-tracking cameras to measure the helicopter pose and eye tracking glasses to determine the operator’s gaze (see Fig). The first step of the investigation was to study the spatial characteristics of human guidance behavior. A mapping technique was developed, taking an ensemble of trajectories, covering the whole region of the task area (see Fig.). In addition, an optimal control model based on minimization of the time needed to reach the goal, i.e., the time-to-go (TTG), was elaborated. The major findings from this mapping technique are: i) the guidance behavior of a trained human subject is coherent in its spatial characteristics, i.e., the speed, heading and time to reach the goal are distributed relatively continuously as a function of the position with respect to the goal state; ii) the optimal control solution based on TTG provides a reasonable model of the spatial and temporal characteristics of trajectory ensembles. The spatial maps and optimal model were formalized under the concept of spatial value function. The next step investigated the organization of guidance behavior in more complex tasks with the hypothesis that humans exploit patterns associated with equivalence relationships between motion and environment as building blocks for behavior organization. Two equivalence relationships inherent to guidance are the symmetries associated with rigid-body motion and the equivalences associated with subgoals. These two relationships were combined under the concept of interaction pattern (IP). The trajectories associated with each IP were then analyzed using system identification techniques. The results from this analysis show that trajectories in the IP are organized in distinct modes that correspond to a startup, coasting, and interception phase. Taken together, the IPs and response modes supply the basic elements for a description of the planning and control hierarchy underpinning human guidance capabilities. They also make it possible to delineate the control and planning functions, their mechanisms and associated information. For planning and organizing behavior the results suggest that guidance behavior is organized spatially in terms of the IPs. More generally, the results support the hypothesis that humans exploit opportunities to organize behavior in such a way that the overall task can be accomplished by implementing a sequence of equivalent behaviors. This involves partitioning the task’s agent-environment dynamics in equivalent regions. The results also suggest that the control functions are organized in a systematic way, where the similar response modes reoccur in each IP. The structures at both levels provide ways to mitigate the various sources of complexity including planning. These results were used to elaborate a hierarchical model (see following Figure). The model comprises three levels: At the top level, guidance behavior is organized in terms of a sequence of subgoals; at the mid level, the behavior is organized in terms of a sequence of dynamic modes; at the bottom level, the closed-loop dynamics, encompassing the vehicle and pilot, can be explained by a linear dynamical system. The control and guidance behavior relies on perceptual functions. The hierarchical model was applied in a preliminary investigation of the perceptual and attentional mechanisms supporting the planning and control functions. The attached figure shows how the principal human visual gaze patterns, i.e., `saccades’, `fixations’ and `smooth pursuits’, classified from the measured gaze data, are distributed in relationship to the trajectory data. These preliminary result show that `fixations’ are concentrated near the subgoal areas and `smooth-pursuits’ are concentrated along the trajectories. This preliminary result provides further support for the significance of the proposed IP and subgoal concepts based on equivalence principles as fundamental unit in the organization and execution of guidance behavior. Applications The final outcome of this grant is a systematic and formal modeling framework of human guidance behavior. The framework is valuable for further scientific investigation of human guidance and control functions and their connection with cognitive neurosciences. Its application to cognitive functions already provides clues about fundamental aspects including attention, information processing, memory and executive functions. Regarding attention, this model makes it possible to understand the segregation between the information necessary for the control in the current IP, and the information necessary to plan the subsequent IP. The framework will help understand individual differences in strategies and further the elaboration of a more detailed understanding concerning the concept of workload. It will also help understand skill acquisition and maintenance, as well as, improve the process of learning from humans. The engineering applications of the framework range from the development of interactive cuing in human-machine systems to the design of novel forms of guidance algorithms for autonomous vehicles, to the design of sensing and perceptual schemes.
