For over two decades, researchers have shown the potential of augmented reality (AR) to transform computer graphics into an everyday extension of human vision that can enhance such diverse applications as medicine, manufacturing, maintenance, smart offices, and navigation. However, there is virtually no use of augmented reality by the public or industry today. The investigators believe that this discrepancy is due largely to the lack of suitable high performance and widely applicable augmented reality displays on which applications can be deployed. The most capable AR displays available today, optical see-through head-mounted displays (HMDs), generally lack four key qualities that prevent their widespread use: wide field-of-view, non-encumbering, support for mutual occlusion, and preservation of most depth cues. The investigators know of no existing or proposed displays that feature all, or even most, of these capabilities. This project takes a radically different approach to optical see-through design that offers the potential to deliver all four missing qualities in a compact form factor that approaches ordinary glasses. The approach relies on a multi-layer display architecture that follows the principles of the emerging field of computational displays - simple optical devices whose functionality and complexity generally lies in software. This project applies existing multi-layer optimization techniques from desktop 3D displays to optical see-through HMDs, while exploring new approaches such as perceptual error metrics, the use multiple layers for occlusion masks, and field of view zone prioritization. This knowledge will be used to build prototype optical see-through displays while handing such challenges as calibration, tracking, computational complexity, and latency. The performance of this approach will be robustly tested and evaluated in simulation, with calibrated cameras, and with human viewers. The target device will transform augmented reality, allowing society to take advantage of the diverse set of applications that have been studied in AR. The proposed design is a radically different approach to optical see-through displays that uses spatial light modulators and software optimization to replace conventional reflective, refractive, and diffractive optics. The ability to produce a focused image on a display placed closer than the eye can accommodate without the use of lenses will be investigated. Sharing of display components for both image formation and occlusion masking will also be researched. The investigators will also explore the use of multi-layer optimization to create multi-focal imagery, prioritize different areas over the viewer's field of view, and facilitate eye tracking.
Broader Impacts: Research and practice have shown the promise of augmented reality to improve such diverse areas as medicine, accessibility, worker efficiency and communications. However, to date there is very little use of augmented reality by the public or industry. This project will lead to a high performance augmented reality display that is badly needed to make the field practical and allow the public to reap the benefits of years of visionary augmented reality research. The science and technology developed in this project will open the use of augmented reality to a wider class of researchers, similar to how the recent development of the commodity depth sensor has permitted new opportunities for scientific inquiry.
