This project develops an integrated Bayesian framework for vision based control of Unmanned Aerial Vehicles (UAVs) through fundamental research in 1) model-based nonlinear state and parameter estimation, 2) intelligent adaptive control, and 3) image processing. We specifically address how real-time video data can be processed with ground-based sensors (and on-board avionics) to extract spatial and situational information (e.g., vehicle state and model parameters). Using only stationary video cameras, information from the sequence of images are integrated with an adaptive controller that transmits actuator commands directly to the UAV. Our research infrastructure consist of an X-Cell-60 R/C helicopter with custom avionics, video cameras on the ground, and a PC ground-station to perform all necessary processing
A key aspect is to go beyond traditional vision based motion estimation and tracking, utilizing new approaches to recursive Bayesian estimation allowing full coupling with the control system. Heuristically, this involves the propagation of probabilistic density estimates for the state (vehicle position, attitude, and velocities) and model parameters (mass, moments of inertia, aerodynamic forces, etc.). The vision components models the ``image likelihood'' and describes the probability of observing the image given the current state. The estimation combines the vision measurements with the dynamic vehicle model in a recursive filtering procedure using a Sigma-Point Filter (SPF) framework. SPF methods are a recent development in machine learning, and are shown to be far superior to standard EKF based estimation approaches.
The intellectual merit of the research contributes to both the individual component areas as well as the integrated whole. The integration of the different components in the proposed manner represents an interdisciplinary new approach, providing new research opportunities and applications in integrated sensing, information processing, and control. Beyond basic research, the broader impact to technology includes the obvious commercial and military applications that can be studied in this controlled environment (e.g. visually assisted vertical take-off and landing for ship board helicopters, or agile maneuvering through urban environments). The core technologies can also be extended to other information technology areas from image tracking and detection, to control of complex biological systems.
