The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.
This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security. This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.
