Perception systems are a key component of modern autonomous cyber-physical systems (CPS), from self-driving vehicles to autonomous robots and drones. For instance, for a self-driving vehicle, perception systems provide functionalities such as estimating the state of the vehicle, building a map of obstacles in its surroundings, and detecting and tracking external objects and pedestrians. As exemplified by recent self-driving car accidents, perception failures can cascade to catastrophic system failures and compromise human safety. Therefore, the development of trustworthy perception systems is paramount to ensure safety and enable adoption of high-integrity and safety-critical CPS applications.
This project lays the foundations of certifiable perception by developing a toolkit of theory, algorithms, and implementations to monitor and drastically reduce subsystem and system-level failures of perception. In particular, this project will (i) develop a new class of certifiable perception algorithms that operate reliably in challenging conditions, are equipped with input-output contracts describing their functionalities, and can formally assert contract satisfaction; (ii) show how to use certifiable algorithms to design contracts for and enable self-supervision of learning-based subsystems; (iii) design system monitors that assert the satisfaction of safety requirements or trigger fail-safe procedures in case of failure; (iv) develop a testbed and real demonstrations of certifiable perception on self-driving car data, focusing on key perception functionalities, such as vehicle localization, environment mapping, and object tracking. This research advances the state of the art in CPS and creates a new research field at the boundary between CPS, robotics and autonomous vehicles, computer vision, machine learning, system-level design and runtime verification.
Certifiable perception will have a transformative impact on a broad range of autonomous CPS where safety, reliability, security, and accountability are key requirements. These include intelligent transportation, supply chain logistics (e.g., last-mile delivery), new aerospace concepts (e.g., autonomous spacecraft, flying taxis, and drones for national security), service and domestic robotics, and collaborative manufacturing. This impact will be enhanced through the release and dissemination of open-source implementations and teaching material, and via demonstrations on real testbeds. The project also boosts K-12, undergraduate, and graduate education, by supporting and actively engaging students in research activities, and through outreach efforts targeting high school students from underrepresented and underserved communities.
This project is in response to the NSF CAREER 20-525 solicitation.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
