The recovery of visual colors from captured images and video signals is among the most prevalent and fundamental problems in digital imaging. This problem affects billions of consumers, and it impacts the quality of the visual signal that these consumers capture, communicate, and display. Millions of image and video cameras, mobile and smart phones, and many other types of visual devices and applications are impacted in a significant way. For example, virtually all consumer cameras are based on an architecture that utilizes a Color Filter Array (CFA), which captures single-color-per-pixel images to reduce cost, size, and power consumption. Hence, one needs to recover the original three color images (Red-Green-Blue) from the captured single-color-per-pixel CFA image. Despite numerous contributions and noticeable progress that has been made in this area, this problem is still largely unsolved.
This project addresses the general problem of the recovery of multiple color channels (RMCC) from limited color information. The project develops a joint rank-minimization sparsity-maximization (RMSM) framework for the recovery of multiple color channels. Rank minimization of matrices, which is a more general framework than compressed sensing (CS) of vectors, provides many powerful tools. The project targets both approaches jointly in novel ways. An important question is how to strike an optimal balance between rank-minimization and sparsity-maximization under a joint framework. Furthermore, this effort designs optimization frameworks for a sparsifying "color"-dictionary paradigm. The notion of utilizing overcomplete, sparsifying "color" dictionaries represents a major departure from prior work. The project is also extending the applications of this research to video demosaicing and visual coding systems.
