Augmented reality (AR), which overlays digital content with the real world around a user, has only recently become available to everyday users. AR applications are finding initial adoption in multiple areas including education, medicine, and gaming. Yet, the technology is currently in its infancy, limited in its ability to adapt to user preferences and environmental conditions, and offering restricted multi-user capabilities. Delivering adaptive multi-user AR experiences poses challenges to existing mobile systems and adaptation algorithms. From a mobile systems perspective, AR devices have strict energy and computing constraints and may experience network failures. Algorithmically, existing adaptation algorithms are often based on distributed machine learning, which is not designed to run on constrained AR devices and may not cope well with environment dynamics. This work will leverage edge computing as a solution for the mobile systems challenges, designing system architectures optimized to provide the similar but not identical user experiences required for multi-user AR. The work will also quantify the performance of existing distributed learning approaches under the constraints of multi-user AR, and will develop algorithms that optimize the resulting performance tradeoffs. The work will finally result in a dataset of AR-related inputs, outputs, and system and network resource utilization characteristics, which will be released publicly for the use of a wider community of researchers to validate their work on AR systems and algorithms.
This project will enable AR capabilities not possible with existing stand-alone or cloud-supported AR, and will extend the scope of applications of machine learning in AR. The work will allow for the development of new AR applications for education, medicine, retail, and gaming that better engage users through delivering more personalized and dynamic user experiences. The developed solutions will be tested in education-specific settings at Duke Lemur Center, where AR will be used to educate visitors to the center about endangered animals and conservation efforts. The research will be integrated into course curricula at Duke and Carnegie Mellon and will be used as the basis for several undergraduate and graduate independent research projects.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
