Augmented reality (AR) technologies -- which overlay digitally generated information on a user's perception of the physical world -- are at the cusp of commercial viability, appearing in platforms like Microsoft HoloLens, Magic Leap, Meta, and automotive windshields. Though these technologies offer great potential benefits, they also raise new and serious computer security and privacy risks. This project focuses on investigating and addressing security and privacy issues due to the virtual output (visual, audio, or haptic feedback) that AR applications generate. Consider, for instance, an AR automotive windshield application that blocks the user's view of pedestrians or road signs with attention-grabbing advertisements, or startles the user with sudden loud advertising jingles; this behavior could have serious physical safety implications. Or consider an AR head-mounted display like the Microsoft HoloLens, in which an application might maliciously draw phobia-inducing objects (like spiders) in the user's visual environment. Combined with integrated education and outreach efforts, this work provides a technical foundation that pushes research and industry towards augmented reality platforms that provide rich functionality and strong security, privacy, and safety properties.
This project explores and addresses several technical challenges with securing AR application outputs. These challenges include: achieving security, privacy, and safety properties simultaneously with high performance and rich functionality; mitigating the risk of physical harm to people and property; and supporting multiple complex applications simultaneously augmenting the user's view of the physical world. The project addresses these challenges by identifying risks due to malicious or buggy AR application output (i.e., visual, audio, or haptic feedback), empirically evaluating these risks in existing and emerging AR platforms (e.g., Microsoft HoloLens and Meta's glasses), and exploring the role of an AR platform's OS in securing virtual output from untrusted applications, while enabling application developers to create rich, compelling AR scenarios. The researchers are designing options for an AR operating system, as well as broader security and safety recommendations for how emerging AR platforms (including car windshields and head-mounted displays) should manage AR output.
