Many common communication situations are over inherently two-way channels, such as telephone systems, digital subscriber lines (DSL), cellular networks, and the Internet. In fact, even `point-to-point' systems, where the end goal is to transfer information in one direction, often give rise to two-way communication scenarios due to the presence of feedback. In such systems, one can receive feedback from the other end of the channel, which can be used to improve the quality of communication. Although feedback is present in many communication systems, and is being used in certain primitive forms as in channel estimation and automatic repeat request, the theory behind its use is far from complete. This research investigates the role of feedback in two-way communication networks and provides architecture-level guidance for designing robust and efficient communication systems. While positive results lead to novel approaches to communication systems design, negative results prevent over-engineering and allow more confidence in simple and modular implementations. At the same time, feedback is a pivotal concept in biological and artificial control systems, learning machines, and communication networks. A deeper understanding of the role of feedback in one area (communication) will lead to a better understanding of the role of feedback in a broader multidisciplinary context.
Concretely, this research focuses on and develops new approaches for tackling problems arising in the following areas: 1) feedback capacity of single-user channels with memory (new coding theorems based on directed information, causal conditioning, and Shannon strategy, as well as the development of concrete schemes for achieving the fundamental limits), 2) multiple-user channels with feedback (emphasis on multiple access channels and broadcast channels: characterization of fundamental limits as well as the construction of practical coding schemes), 3) capacity region of the two-way channel such as the Blackwell-Shannon binary multiplying channel (dynamic programming and infinite-dimensional convex optimization), 4) robust feedback coding techniques under channel uncertainty (universal decoding schemes), and 5) reliable communication with noisy feedback (new perspectives on cross-layer design of channel codes and network protocols). Massey's directed information takes the role of Shannon's mutual information in many feedback communication problems. Thus the role of directed information is investigated as a fundamental notion in general causal inference problems. Examples include gambling in horse-race markets with causal side information and its dual in source coding.
