This Small Business Innovation Research (SBIR) Phase I proposal aims to create a low bandwidth streaming algorithm for high definition (HD) videoconferencing. The ultimate goal is a software system that achieves less than 150 msec one-way end-to-end delay (the typical delays of telephone) for 1280x720 @ 30 fps video at 512 Kbps (the typical upload speeds of high-end consumer broadband). Most videoconferencing systems today stream video at 1/10 of HD resolution. At such resolution, it is often difficult to see facial expressions.
Commercial HD products are currently only available in hardware. These systems all require network speeds beyond the limits of broadband. This research aims to develop videoconferencing software for broadband. Faces and whiteboards are the two most common elements of interest in a videoconference. This research will combine real-time network sensing and the domain knowledge of how people perceive facial expressions, whiteboards, and nondescript background to create a unique compression/streaming algorithm. The input video is segmented into different regions. The whiteboard and background will be enhanced for sharpness using novel super resolution techniques. Two core technologies will be developed: 1) perceptual resolution enhancement of whiteboard and background, and 2) network adaptive HD streaming. Although the concepts just described have been explored in principle by thousands of researches over the last few decades, previous efforts did not result in any HD products.
