×

A strong limit theorem for functions of continuous random variables and an extension of the Shannon-McMillan theorem. (English) Zbl 1145.60309

Summary: By means of the notion of likelihood ratio, the limit properties of the sequences of arbitrary-dependent continuous random variables are studied, and a kind of strong limit theorems represented by inequalities with random bounds for functions of continuous random variables is established. The Shannon-McMillan theorem is extended to the case of arbitrary continuous information sources. In the proof, an analytic technique, the tools of Laplace transform, and moment generating functions to study the strong limit theorems are applied.

MSC:

60F15 Strong limit theorems
94A15 Information theory (general)
60E10 Characteristic functions; other transforms
PDFBibTeX XMLCite
Full Text: DOI EuDML

References:

[1] R. G. Laha and V. K. Rohatgi, Probability Theory, Wiley Series in Probability and Mathematical Statistic, John Wiley & Sons, New York, NY, USA, 1979. · Zbl 0409.60001
[2] P. Billingsley, Probability and Measure, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York, NY, USA, 2nd edition, 1986. · Zbl 0649.60001
[3] W. Liu, “Relative entropy densities and a class of limit theorems of the sequence of m-valued random variables,” The Annals of Probability, vol. 18, no. 2, pp. 829-839, 1990. · Zbl 0711.60026 · doi:10.1214/aop/1176990860
[4] W. Liu, “A class of strong deviation theorems and Laplace transform methods,” Chinese Science Bulletin, vol. 43, no. 10, pp. 1036-1041, 1998.
[5] W. Liu and Y. Wang, “A strong limit theorem expressed by inequalities for the sequences of absolutely continuous random variables,” Hiroshima Mathematical Journal, vol. 32, no. 3, pp. 379-387, 2002. · Zbl 1016.60035
[6] W. Yang, “Some limit properties for Markov chains indexed by a homogeneous tree,” Statistics & Probability Letters, vol. 65, no. 3, pp. 241-250, 2003. · Zbl 1068.60045 · doi:10.1016/j.spl.2003.04.001
[7] W. Liu, Strong Deviation Theorems and Analytic Method, Science Press, Beijing, China, 2003.
[8] J. L. Doob, Stochastic Processes, John Wiley & Sons, New York, NY, USA, 1953. · Zbl 0053.26802
[9] W. Liu and J. Wang, “A strong limit theorem on gambling systems,” Journal of Multivariate Analysis, vol. 84, no. 2, pp. 262-273, 2003. · Zbl 1016.60033 · doi:10.1016/S0047-259X(02)00054-4
[10] T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications, John Wiley & Sons, New York, NY, USA, 1991. · Zbl 0762.94001
[11] C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379-423, 623-656, 1948. · Zbl 1154.94303
[12] P. H. Algoet and T. M. Cover, “A sandwich proof of the Shannon-McMillan-Breiman theorem,” The Annals of Probability, vol. 16, no. 2, pp. 899-909, 1988. · Zbl 0653.28013 · doi:10.1214/aop/1176991794
[13] A. R. Barron, “The strong ergodic theorem for densities: generalized Shannon-McMillan-Breiman theorem,” The Annals of Probability, vol. 13, no. 4, pp. 1292-1303, 1985. · Zbl 0608.94001 · doi:10.1214/aop/1176992813
[14] K. L. Chung, “A note on the ergodic theorem of information theory,” The Annals of Mathematical Statistics, vol. 32, no. 2, pp. 612-614, 1961. · Zbl 0115.35503 · doi:10.1214/aoms/1177705069
[15] J. C. Kieffer, “A simple proof of the Moy-Perez generalization of the Shannon-McMillan theorem,” Pacific Journal of Mathematics, vol. 51, pp. 203-206, 1974. · Zbl 0281.94007 · doi:10.2140/pjm.1974.51.203
[16] J. C. Kieffer, “A counterexample to Perez’s generalization of the Shannon-McMillan theorem,” The Annals of Probability, vol. 1, no. 2, pp. 362-364, 1973. · Zbl 0262.94017 · doi:10.1214/aop/1176996994
[17] B. McMillan, “The basic theorems of information theory,” The Annals of Mathematical Statistics, vol. 24, no. 2, pp. 196-219, 1953. · Zbl 0050.35501 · doi:10.1214/aoms/1177729028
[18] W. Liu and W. Yang, “An extension of Shannon-McMillan theorem and some limit properties for nonhomogeneous Markov chains,” Stochastic Processes and Their Applications, vol. 61, no. 1, pp. 129-145, 1996. · Zbl 0861.60042 · doi:10.1016/0304-4149(95)00068-2
[19] W. Liu and W. Yang, “The Markov approximation of the sequences of N-valued random variables and a class of small deviation theorems,” Stochastic Processes and Their Applications, vol. 89, no. 1, pp. 117-130, 2000. · Zbl 1051.94005 · doi:10.1016/S0304-4149(00)00016-8
[20] R. M. Gray, Entropy and Information Theory, Springer, New York, NY, USA, 1990. · Zbl 0722.94001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.