This is the first year funding of a three-year continuing award. The project investigates on-chip analog and mixed mode computation for an intermediate sensory signal representation before the information is read out from the sensor chip. By appropriate information encoding, such representation will be robust to limitations of the sensing and readout process, thus enabling efficient extraction of useful environmental information. On-chip computation enables a sensor to make partial decisions on-chip and to use those decisions to create an optimal signal representation and robust extraction of sensory information. This project will focus on image sensors. From a theoretical point of view, image sensors are interesting because they are scalable parallel systems that require fine-grain, distributed computation and global data communication among a large number of sites. The necessity to bring together data from a large number of processors/sites quickly saturates communication connectivity and adversely affects computing efficiency in large parallel systems. The aim of this project is not to miniaturize conventional image processing on-chip, but rather to obtain information about the environment that is not obtainable if computation is not performed on the sensory level. From a practical point of view, this project focuses on the ability of image sensors to adapt to high and low dynamic range scenes. Such scenes routinely cause conventional sensors to fail; yet such scenes are omnipresent in everyday imaging applications. The PI's group will build several high resolution CMOS line and area computational image sensors to test our signal encoding techniques. Ultimately chips will be tested in an ongoing robotic application at the CMU Robotics Institute. One candidate application is a visual guidance of fully autonomous or robot-assisted wheelchair for severely disabled users who are able to provide only high-level control to the system. The primary significance of this research is toward alleviating the problem of global data aggregation and communication from large and scalable parallel systems. The secondary significance of this research is in its practical application to visual perception systems and its superior information extraction ability.
