Many communication and compression scenarios involve the presence of Side Information (SI) on the state of the channel through which communication is to take place, or on the information source which is to be communicated. The role and potential benefit of such SI is a central theme in information theory. In ways that are well understood for various source and channel coding systems, SI can be a valuable resource, resulting in significant performance boosts relative to the case where it is absent. In the problems studied thus far, however, the lack or availability of the SI, and its quality, are a given. This research is geared towards characterizing fundamental limits ?and devising guidelines for the construction of practical schemes? in scenarios involving systems that can take actions affecting the availability, quality, or nature of the SI. A central component of this research is the study of control theoretic notions such as action and actuation from an information theoretic perspective. Specifically, we study source coding scenarios involving the presence of side information, when the system can take actions that affect the availability, quality, or nature of the SI. We begin by extending the Wyner-Ziv problem of source coding with decoder side information to the case where the decoder is allowed to choose actions affecting the SI. We consider also settings where actions are taken by the encoder(s). In a parallel vein, we study channels with action-dependent states: Given the messages to be communicated, the transmitter chooses an action sequence that affects the formation of the channel states, and then creates the channel input sequence based on the state sequence. We characterize the capacity of such a channel both for the case where the channel inputs are allowed to depend non-causally on the state sequence and the case where they are restricted to causal dependence. Actions may have costs that are commensurate with the quality of the SI they yield, and an overall per-symbol cost constraint may be imposed. We characterize the achievable tradeoffs between rate, distortion, power and cost in such source and channel coding systems, in both point-to-point and multi-terminal settings. Our models cover various new information processing scenarios ranging from sensing and data acquisition to coding for computer memories with a ?rewrite? option.
This project was dedicated to the introduction and study of notions of action and actuation in point-to-point and multi-terminal channel coding (communication) and source coding (compression) problems. This exploration has given rise to various new and timely coding scenarios arising naturally in wireless communications and storage for computer memories with defects, where the availability and quality of channel state information is commensurate with expenditure of system resources. The significance of the findings is not only in formulating and answering fundamentally new communication theoretic questions of timely practical relevance, but also in that some of the answers are rather surprising and counterintuitive. For example, the notion of ‘probing capacity’ was introduced and it has been shown that when the transmitter has the freedom to choose which of the channel states to observe, he can achieve the same rate of reliable communication as if all the states were observed, even when allowed to observe only a small fraction of these states. The results of this work are guiding the construction of new communication systems, where channel state information is increasingly recognized and treated as a resource to be acquired upon expenditure of system resources. Another related line of work done under the auspices of this project pertains to new feedback communication settings, where transmitter and receiver can take decisions that drive the acquisition of feedback. These models capture many practical and timely scenarios where cost constraints for transmitters and receivers govern the quality or availability of any information used in the communication process, including channel state information and feedback. The capacity (maximal achievable rate of reliable communication) of this channel, and the form of the schemes that achieve it, have been characterized. These findings will likely have direct bearing on the design of future communication systems.
