The goal of this project is to enable multiple co-robots to map and understand the environment they are in to efficiently collaborate among themselves and with human operators in education, medical assistance, agriculture, and manufacturing applications. The first distinctive characteristic of this project is that the environment will be modeled semantically, that is, it will contain human-interpretable labels (e.g., object category names) in addition to geometric data. This will be achieved through a novel, robust integration of methods from both computer vision and robotics, allowing easier communications between robots and humans in the field. The second distinctive characteristic of this project is that the increased computation load due to the addition of human-interpretable information will be handled by judiciously approximating and spreading the computations across the entire network. The novel developed methods will be evaluated by emulating real-world scenarios in manufacturing and for search-and-rescue operations, leading to potential benefits for large segments of the society. The project will include opportunities for training students at the high-school, undergraduate, and graduate levels by promoting the development of marketable skills.
The project will advance the state of the art in robust semantic mapping from multiple robots by 1) developing a new optimization framework that can handle large, dynamic, uncertain environments under significant measurement errors, 2) explicitly allowing and studying interactions and information exchanges with humans with an hybrid discrete-continuous extension of the optimization framework, and 3) allowing an intelligent use and sharing of the limited computational resources possessed by the network of co-robots as a whole by enabling approximations and balancing of the computations. These developments will be driven by two particular case studies: a job-shop (small factory) scenario, where robots and fixed cameras are used to track and assist human workers during production and assembly of parts; and a classic search-and-rescue scenario, where operators use an heterogeneous team of robots to quickly assess damages and to discover survivors. These two applications, when considered together, highlight all the limitations of the currently prevalent geometric mapping solutions, and will be used as benchmarks for the project's results.
