The ability to collect videos can revolutionize interactions with the world by enabling powerful virtual reality video applications in education, tourism, tele-presence and others. These new applications involve processing and serving 360-degree stereoscopic videos, which require a dramatic improvement in technology to manage and process the massive-scale visual data necessary for truly immersive experiences. Systems that support VR video also represent an excellent educational tool as students can experience scenes in a truly immersive way and hence better convey content. This project builds a visual cloud that provides seamless access to a new database management system with hardware acceleration and edge computation that enables the efficient and real-time management of massive-scale image data and virtual reality (VR) applications built on top of it. The project develops a new hardware and software stack for VR data processing with execution in public clouds. The stack includes a new storage manager that significantly increases data ingest and retrieval throughputs for multidimensional array data compared with existing systems, as motivated by the extreme needs of VR applications. The storage manager utilizes novel hardware technologies (non-volatile memory) and provides novel approximate and multi-resolution data storage capabilities. The project also develops a new runtime system for high-throughput and large-scale array processing by developing a new API for expressing VR pipelines as a graph of user-defined functions, a library of specialized implementations of known VR algorithms for different types of hardware (CPU, GPU, FPGA, and 3D XPoint), and associated optimizers and schedulers. Finally, the project develops new techniques to enable real-time VR applications. They include an FPGA-based acceleration platform for real-time VR video processing and algorithms and software components for prefetching and caching VR data close to the viewers and processing that data in the viewing device. http://visualcloud.cs.washington.edu
