Current methods of image and video coding are effective largely because they capitalize on low-level aspects of the human visual system (HVS). The single most predominant strategy is to place the errors into regions which can better hide the compression artifacts, an approach which can be guided by computational models of early/low-level HVS processing. The inherent assumption in this approach is that the consumer is looking for the distortion in the presence of the image. However, in actuality, the consumer is looking at the image in the presence of possible distortion, which is a fundamentally different perceptual task that requires a fundamentally different HVS model. Next-generation coding schemes which can take into account higher-level aspects such as content-adaptive masking; perceptual importance across space, frequency, and time; and elements of cognition; have the potential to dramatically reduce storage and bandwidth requirements while maximizing visual quality and the overall multimedia experience. In this project, the investigator researches how compression artifacts influence the HVS's ability to process and interpret images and video. Three main areas are investigated: (1) new models of visual masking which take into account image recognition; (2) appearance-preserving strategies of data quantization; and (3) analysis and quantization strategies which honor rules of visual cognition derived from quality-rating experiments coupled with eye-tracking. This research is integrated with an educational component that promotes student development in applying knowledge of human vision to engineering problems. Two new interdisciplinary graduate-level courses, an interdisciplinary summer workshop, and undergraduate research projects and curriculum reform are made available to students. Two multimedia-driven competitions that expose K-12 students and undergraduates to image-processing research are also made available.
