This project studies robots that utilize the nearby and available physical objects to perform tasks, such as building a bridge out of miscellaneous rubble in a disaster area. Resourceful robots have the potential to enable many new applications in emergency response, healthcare, and manufacturing, which will improve the welfare, security, and efficiency of the overall population. The research investigates how the patterns between multiple senses, such as vision, sound, and touch, will help teach the robot to solve interaction tasks without needing a human teacher, which is expected to improve the flexibility and versatility of autonomous robots. The project will provide research and educational opportunities for both graduate and undergraduate students in computer science and mechanical engineering. Outcomes from this research will translate into new educational materials in computer vision, machine learning, and robotics.
This research investigates robots that interact with realistic environments in order to learn reusable representations for navigation and manipulation tasks. While there has been significant advancements leveraging machine learning for computer vision and robotics problems, a central challenge in both fields is generalizing to the realistic complexity and diversity of the physical world. Although simulation has proved instrumental in developing platforms for machine interaction, the unconstrained world is vast, making it computationally difficult to simulate. Instead, the investigators aim to capitalize on the inherent structure of physical environments through the natural synchronization of modalities and context to efficiently learn self-supervised representations and policies for interaction with unconstrained environments. The investigators also plan several evaluations to analyze the generalization capabilities of such algorithms.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
