We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Library & Information Science/Directed Information on Abstract Spaces: Properties and Variational Equalities
PeriodicalsLibrary & Information Science

Directed Information on Abstract Spaces: Properties and Variational Equalities

arXiv2013

Checking your access to this record.

Abstract & Overview

Directed information or its variants are utilized extensively in the characterization of the capacity of channels with memory and feedback, nonanticipative lossy data compression, and their generalizations to networks. In this paper, we derive several functional and topological properties of directed information for general abstract alphabets (complete separable metric spaces) using the topology of weak convergence of probability measures. These include convexity of the set of consistent distributions, which uniquely define causally conditioned distributions, convexity and concavity of directed information with respect to the sets of consistent distributions, weak compactness of these sets of distributions, their joint distributions and their marginals. Furthermore, we show lower semicontinuity of directed information, and under certain conditions we also establish continuity of directed information. Finally, we derive variational equalities for directed information, including sequential versions. These may be viewed as the analogue of the variational equalities of mutual information (utilized in Blahut-Arimoto algorithm). In summary, we extend the basic functional and topological properties of mutual information to directed information. These properties are discussed in the context of extremum problems of directed information.

Cite this record

Charalambos D. Charalambous, Photios A. Stavrou (2013). Directed Information on Abstract Spaces: Properties and Variational Equalities. arXiv. https://library.shubhaarti.com/content/faed0e67-ec7f-4408-93ce-0888b4a7315a

Copy in another style

Charalambous, C. D., & Stavrou, P. A. (2013). Directed Information on Abstract Spaces: Properties and Variational Equalities. arXiv. https://library.shubhaarti.com/content/faed0e67-ec7f-4408-93ce-0888b4a7315a
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.