Autonomous robot systems offer tremendous potential for transforming various industry sectors, including transportation, construction, mining, and agriculture. The operational conditions in these domains, however, are unstructured and dynamically changing. This poses a major challenge for current robot system designs as they depend on static offline environment models and rigid dynamics models that do not improve with operational experience. The impact of autonomous systems in these domains is also limited by hand-designed safety rules that fail to account for the complexity and uncertainty of real-world operation. This project will develop new theoretical and algorithmic tools for advancing the ability of autonomous systems to comprehend their surroundings online from sensory observations and adapt their operation safely in response to changing conditions. On the educational front, the project aims to increase the participation of underrepresented undergraduate students in education and research activites related to robot autonomy in human environments and develop new talent in the STEM fields, which will be critical for the future of the U.S. economy.
The key innovations of the project include techniques for online inference of object shapes and robot dynamics models from sensory observations as well as control design for the learned robot dynamics, subject to safety constraints from the observed objects. First, an implicit surface model of object shape, compactly encoded with a latent feature vector is proposed. An online optimization algorithm to estimate the object poses, shapes, and robot motion jointly is developed. This algorithm enables semantic environment understanding and specification of safety constraints for autonomous navigation. Second, the spectral properties of the Koopman operator and the versatility of Bayesian neural networks are leveraged to design self-supervised robot dynamics learning algorithms. These algorithms provide an adaptive way of estimating robot dynamics from online data, while relying on approximation error bounds to guarantee that the control design satisfies the safety constraints provided by the perceptual system. These innovations will enable autonomous robot operation in unknown environments that is adaptable, due to the use of learned robot and object models, and safe, due to the use of perception- and uncertainty-aware constraints in the control design.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
