Ergodic theory, entropy, and coding problems of information theory. (English) Zbl 0542.94006

Šujan, Štefan

Kybernetika 19, Suppl. No. 1-4, 67 pp. (1983).

This paper is a survey of the development of entropy methods in the ergodic theory of measure theoretic dynamical systems, during the period 1970–1980. The author’s goal is to make more accessible to the information theorist the recent work in this area, by translating results into the language of information theory. This paper achieves its purpose, is well written and provides an extensive bibliography.

Reviewer: C.L.Byrne

MSC:

94A17	Measures of information, entropy
94A15	Information theory (general)
94-02	Research exposition (monographs, survey articles) pertaining to information and communication theory
37A25	Ergodicity, mixing, rates of mixing
28D20	Entropy and other invariants

Keywords:

coding; survey; entropy; ergodic theory of measure theoretic dynamical systems

References:

[1]	M. A. Ackoglu A. del Junco, and M. Rahe: Finitary codes between Markov processes. Z. Wahrsch. verw. Gebiete 47 (1979), 305-314. · Zbl 0403.28017 · doi:10.1007/BF00535166
[2]	R. L. Adler W. Goodwyn, and B. Weiss: Equivalence of topological Markov shifts. Israel J. Math. 27 (1977), 49-63. · Zbl 0362.54034 · doi:10.1007/BF02761605
[3]	R. L. Adler A. G. Konheim, and M. H. McAndrew: Topological entropy. Trans. Amer. Math. Soc. 114 (1965), 309-319. · Zbl 0127.13102 · doi:10.2307/1994177
[4]	R. L. Adler, B. Marcus: Topological entropy and equivalence of dynamical systems. Memoirs Amer. Math. Soc. 219 (1979). · Zbl 0412.54050
[5]	R. L. Adler, B. Weiss: Similarity of the automorphisms of the torus. Memoirs Amer. Math. Soc. 98 (1970). · Zbl 0195.06104
[6]	V. M. Alekseev: Symbolic Dynamics. (in Russian). Math. Institute, AN USSR, Kiev 1976.
[7]	V. M. Alekseev, M. V. Jakobson: Symbolic dynamics and hyperbolic dynamical systems. (in Russian). Supplement to R. Bowen: Methods of Symbolic Dynamics (in Russian). Mir, Moskva 1979, pp. 196-240.
[8]	R. B. Ash: Information Theory. J. Wiley, New York 1965. · Zbl 0141.34904
[9]	T. Berger: Rate Distortion Theory. Prentice Hall, Englewood Cliff’s 1971.
[10]	T. Berger: Information singular processes. IEEE Trans. Inform. Theory IT-20 (1975), 502-511.. · Zbl 0312.94008 · doi:10.1109/TIT.1975.1055438
[11]	P. Billingsley: Ergodic Theory and Information. J. Wiley, New York-London-Sydney 1965. · Zbl 0141.16702
[12]	P. Billingsley: Convergence of Probability Measures. J. Wiley, New York-London-Sydney-Toronto 1968. · Zbl 0172.21201
[13]	R. E. Blahut: Computation of channel capacity and rate-distortion functions. IEEE Trans. Inform. Theory IT-18 (1972), 460-473. · Zbl 0247.94017 · doi:10.1109/TIT.1972.1054855
[14]	H. Blasbalg, R. van Blerkom: Message compression. IRE Trans. Space Electron. Telemech. 1962, 228-338.
[15]	J. R. Blum, D. L. Hanson: On invariant probability measures I. Pacific J. Math. 10 (1960), 1125-1240.
[16]	J. R. Blum, D. L. Hanson: On the isomorphism problem for Bernoulli schemes. Bull. Amer. Math. Soc. 63 (1963), 221-223. · Zbl 0121.13601 · doi:10.1090/S0002-9904-1963-10924-6
[17]	R. Bowen: Symbolic dynamics for hyperbolic systems. Amer. J. Math. 95 (1973), 429-459. · Zbl 0257.54042
[18]	R. Bowen: Topological entropy for non-compact sets. Trans. Amer. Math. Soc. 184 (1973), 413-423.
[19]	R. Bowen: Smooth partitions of Anosov diffeomorphisms are weak Bernoulli. Israel J. Math. 27(1975), 95-100. · Zbl 0315.58020 · doi:10.1007/BF02760788
[20]	R. Bowen: Equilibrium States and the Ergodic Theory of Anosov Diffeomorphisms. (Lecture Notes in Mathematics 470.) Springer-Verlag, Berlin-Heidelberg-New York 1975. · Zbl 0308.28010
[21]	R. Bowen, D. Ruelle: The ergodic theory of Axiom A flows. Invent. Mathematicae 29 (1975), 181-202. · Zbl 0311.58010 · doi:10.1007/BF01389848
[22]	R. C. Bradley, Jr.: On the strong mixing and weak Bernoulli conditions. Z. Wahrsch. verw. Gebiete 51 (1980), 49-54. · Zbl 0396.60038 · doi:10.1007/BF00533816
[23]	A. A. Brudno: Entropy and algorithmic complexity of trajectories of a dynamical system. (in Russian). Preprint, VNIISI, Moskva 1980.
[24]	E. Coven, M. Paul: Endomorphisms of irreducible shifts of finite type. Math. Systems Theory 8(1974), 165-175. · Zbl 0309.54032
[25]	L. D. Davisson: Universal noiseless coding. IEEE Trans. Inform. Theory IT-19 (1973), 783-795. · Zbl 0283.94005 · doi:10.1109/TIT.1973.1055092
[26]	L. D. Davisson, R. M. Gray: A simplified proof of the sliding-block source coding theorem and its universal extension. Proc. Int. Conf. on Communication, Vol. 2, pp. 34.4.1 - 34.4.5. Toronto, Canada 1978.
[27]	A. del Junco, M. Rahe: Finitary codings and weak Bernoulli partitions. Proc. Amer. Math. Soc. 75 (1979), 259-364. · Zbl 0417.28011 · doi:10.2307/2042753
[28]	M. Denker: Finite generators for ergodic measure-preserving transformations. Z. Wahrsch. verw. Gebiete 29 (1974), 45-55. · Zbl 0278.28006 · doi:10.1007/BF00533186
[29]	M. Denker: Generators and almost topological isomorphisms. Astérisque 59 (1977), 23-35. · Zbl 0384.28013
[30]	M. Denker C. Grillenberger, and K. Sigmund: Ergodic Theory on Compact Spaces. (Lecture Notes in Mathematics 527.) Springer-Verlag, Berlin-Heidelberg-New York 1976. · Zbl 0328.28008 · doi:10.1007/BFb0082364
[31]	M. Denker, M. Keane: Almost topological dynamical systems. Israel J. Math. 34 (1979), 139-160. · Zbl 0441.28008 · doi:10.1007/BF02761830
[32]	M. Denker, M. Keane: Finitary codes and the law of the iterated logarithm. Z. Wahrsch. verw. Gebiete 52 (1980), 321-331. · Zbl 0417.60040 · doi:10.1007/BF00538896
[33]	J. G. Dunham: Abstract alphabet sliding-block entropy compression coding with a fidelity criterion. Ann. Probab. 8 (1980), 1085-1092. · Zbl 0452.94009 · doi:10.1214/aop/1176994570
[34]	R. Fellgett, W. Parry: Endomorphisms of a Lebesgue space II. Israel J. Math. 21 (1975), 167-172. · Zbl 0305.28009 · doi:10.1112/blms/7.2.151
[35]	B. M. Fitingoff: Optimal coding in case of unknown and changing message statistics. (in Russian). Problemy Peredachi Informacii 2 (1966), 3-11.
[36]	B. M. Fitingoff: The compression of discrete information. (in Russian). Problemy Peredachi Informacii 3 (1967), 28-36. · Zbl 0295.94042
[37]	R. J. Fontana R. M. Gray, and J. C. Kieffer: Asymptotically mean stationary channels. IEEE Trans. Inform. Theory IT-27 (1981), 308-316. · Zbl 0483.94010 · doi:10.1109/TIT.1981.1056348
[38]	N. Friedman, D. S. Ornstein: On the isomorphism of weak Bernoulli transformations. Adv. in Math. 5 (1970), 365-394. · Zbl 0203.05801 · doi:10.1016/0001-8708(70)90010-1
[39]	R. G. Gallager: Information Theory and Reliable Communication. J. Wiley, New York 1968. · Zbl 0198.52201
[40]	F. R. Gantmacher: The Theory of Matrices. Vols. I and II. Chelsea, New York 1959. · Zbl 0085.01001
[41]	R. M. Gray: Sliding-block source coding. IEEE Trans. Inform. Theory IT-21 (1975), 357-368. · Zbl 0308.94015 · doi:10.1109/TIT.1975.1055422
[42]	R. M. Gray, L. D. Davisson: The ergodic decomposition of stationary discrete random processes. IEEE Trans. Inform. Theory IT-20 (1974), 625 - 636. · Zbl 0301.94027 · doi:10.1109/TIT.1974.1055287
[43]	R. M. Gray, L. D. Davisson: Source coding theorems without the ergodic assumption. IEEE Trans. Inform. Theory IT-20 (1974), 502-516. · Zbl 0301.94026 · doi:10.1109/TIT.1974.1055248
[44]	R. M. Gray, J. C. Kieffer: Mutual information rate, distortion, and quantization in metric spaces. IEEE Trans. Inform. Theory IT-26 (1980), 412-422. · Zbl 0452.94010 · doi:10.1109/TIT.1980.1056222
[45]	R. M. Gray, J. C. Kieffer: Asymptotically mean stationary measures. Ann. Probab. (1980), 962-973. · Zbl 0447.28014 · doi:10.1214/aop/1176994624
[46]	R. M. Gray D. L. Neuhoff, and J. K. Omura: Process definitions of distortion-rate function and source coding theorems. IEEE Trans. Inform. Theory IT-21 (1975), 524-532. · Zbl 0313.94008 · doi:10.1109/TIT.1975.1055440
[47]	R. M. Gray D. L. Neuhoff, and D. S. Ornstein: Non block source coding with a fidelity criterion. Ann. Probab. 3 (1975), 478-491. · Zbl 0313.94006 · doi:10.1214/aop/1176996354
[48]	R. M. Gray D. L. Neuhoff, and P. C. Shields: A generalization of Ornstein’s d-distance with applications to information theory. Ann. Probab. 3 (1975), 315 - 328. · Zbl 0304.94025 · doi:10.1214/aop/1176996402
[49]	R. M. Gray, D. S. Ornstein: Sliding-block joint source/noisy channel coding theorems. IEEE Trans. Inform. Theory IT-22 (1976), 683-690. · Zbl 0348.94019 · doi:10.1109/TIT.1976.1055642
[50]	R. M. Gray, D. S. Ornstein: Block coding for discrete stationary d-continuous noisy channels. IEEE Trans. Inform. Theory IT-25 (1979), 292-306. · Zbl 0398.94013 · doi:10.1109/TIT.1979.1056045
[51]	R. M. Gray D. S. Ornstein, and R. L. Dobrushin: Block synchronization, sliding-block coding, invulnerable sources, and zero-error codes for discrete noisy channels. Ann. Probab. 8 (1980), 639-674. · Zbl 0453.94010 · doi:10.1214/aop/1176994658
[52]	C. Grillenberger, U. Krengel: On marginaldistributionsandisomorphisms of stationary processes. Math. Z. 149, (1976), 131-154. · Zbl 0313.60029
[53]	B. Hajek: Information-singularity and recoverability of random processes. IEEE Trans. Inform. Theory IT-28 (1983), 422-429. · Zbl 0478.94014 · doi:10.1109/TIT.1982.1056513
[54]	P. R. Halmos: Measure Theory. D. Van Nostrand, Princeton N. J. 1950. · Zbl 0040.16802
[55]	P. R. Halmos: Lectures on Ergodic Theory. Chelsea, New York 1953. · Zbl 0073.09302
[56]	G. Hansel, J. P. Raoult: Ergodicité, uniformité et unique ergodicité. Indiana Univ. Math. J. 23 (1973), 221-237. · Zbl 0275.28017
[57]	R. Heim: On the algorithmic foundations of information theory. IEEETrans. Inform. Theory 1T-25 (1979), 557-566. · Zbl 0419.94002 · doi:10.1109/TIT.1979.1056080
[58]	R. I. Jewett: The prevalence of uniquely ergodic systems. J. Math. and Mech. 19 (1970), 717-729. · Zbl 0192.40601
[59]	M. Keane: Coding problems in ergodic theory. Proc. Int. Conf. on Math. Physics. Camerino, Italy, 1974.
[60]	M. Keane, M. Smorodinsky: A class of finitary codes. Israel J. Math. 26 (1977), 352-371. · Zbl 0357.94012 · doi:10.1007/BF03007652
[61]	M. Keane, M. Smorodinsky: Bernoulli schemes of the same entropy are finitarily isomorphic. Ann. Math. 109 (1979). 397-406. · Zbl 0405.28017 · doi:10.2307/1971117
[62]	M. Keane, M. Smorodinsky: The finitary isomorphism theorem for Markov shifts. Bull. (New Series) Amer. Math. Soc. 1 (1979), 436-438. · Zbl 0407.28010 · doi:10.1090/S0273-0979-1979-14633-0
[63]	A. I. Khinchine: Mathematical Foundations of Information Theory. Dover, New York 1957.
[64]	J. C. Kieffer: On approximation of stationary measures by periodic and ergodic measures. Ann. Probab. 2 (1974), 530-534. · Zbl 0287.60033
[65]	J. C. Kieffer: A generalized Shannon-McMillan theorem for the action of an amenable group on a probability space. Ann. Probab. 3 (1975), 1031 - 1037. · Zbl 0322.60032 · doi:10.1214/aop/1176996230
[66]	J. C. Kieffer: Block coding for an ergodic source relative to a zero-one valued fidelity criterion. IEEE Trans. Inform. Theory IT-24 (1978), 432-438. · Zbl 0385.94011 · doi:10.1109/TIT.1978.1055914
[67]	J. C. Kieffer: A unified approach to weak universal source coding. IEEE Trans. Inform. Theory IT-24 (1978), 674-682. · Zbl 0394.94012 · doi:10.1109/TIT.1978.1055960
[68]	J. C. Kieffer: On the minimum rate for strong universal block coding of a class of ergodic sources. IEEE Trans. Inform. Theory IT-26 (1980), 693-702. · Zbl 0473.94007 · doi:10.1109/TIT.1980.1056260
[69]	J. C. Kieffer: On the transmission of Bernoulli sources over stationary channels. Ann. Probab. 8 (1980), 942-961. · Zbl 0452.94012 · doi:10.1214/aop/1176994623
[70]	J. C. Kieffer: On coding a stationary process to achieve a given marginal distribution. Ann. Probab. 8 (1980), 131-141. · Zbl 0426.60036 · doi:10.1214/aop/1176994829
[71]	J. C. Kieffer: Extensions of source codirig theorems for block codes to sliding-block codes. IEEE Trans. Inform. Theory IT-26 (1970), 679-692. · Zbl 0473.94006
[72]	J. C. Kieffer: Stationary coding over stationary channels. Z. Wahrsch. verw. Gebiete 56 (1981), 113-136. · Zbl 0444.94007 · doi:10.1007/BF00531977
[73]	J. C. Kieffer: Block coding for weakly continuous channels. IEEE Trans. Inform. Theory IT-27 (1981), 721-727. · Zbl 0473.94004 · doi:10.1109/TIT.1981.1056422
[74]	J. C. Kieffer: Perfect transmission over a discrete memoryless channel requires infinite expected coding time. J. Combin. Inform. System Sci. 5 (1980), 317-322. · Zbl 0475.94007
[75]	J. C. Kieffer: Sliding-block coding for weakly continuous channels. IEEE Trans. Inform. Theory IT-28 (1982), 2-10. · Zbl 0473.94005
[76]	J. C. Kieffer: Characterizations of d-total boundedness for classes of B sources. IEEE Trans. Inform. Theory IT-28 (1982), 26-35. · Zbl 0473.94009 · doi:10.1109/TIT.1982.1056451
[77]	J. C. Kieffer: On obtaining a stationary process isomorphic to a given process with a desired distribution. Preprint, Univ. of Missouri at Rolla, 1982. · Zbl 0518.60057 · doi:10.1007/BF01605487
[78]	J. C. Kieffer: Generators with prescribed marginals for nonergodic automorphisms. Lecture presented at the 9th Prague Conf. Inform. Theory, Prague, June 1982. · Zbl 0547.28012
[79]	J. C. Kieffer, M. Rahe: Selecting universal partitions in ergodic theory. Ann. Probab. 9 (1981), 705-709. · Zbl 0464.60036 · doi:10.1214/aop/1176994379
[80]	A. N. Kolmogorov: A new metric invariant of transitive dynamical systems and automorphisms of Lebesgue spaces. (in Russian). Doklady AN SSSR 119 (1958), 862-864. · Zbl 0083.10602
[81]	A. N. Kolmogorov: The three approaches to the definition of the concept ”amount of information”. (in Russian). Problemy Peredachi Informacii 5 (1965), 3 - 7. · Zbl 0271.94018
[82]	I. P. Kornfeľd, Ya. G. Sinai, and S. V. Fomin: Ergodic Theory. (in Russian). Nauka, Moskva 1980.
[83]	U. Krengel: Recent results on generators in ergodic theory. Trans. 6th Conf. Inform. Theory etc, Academia, Prague 1973, 465 - 482. · Zbl 0318.94024
[84]	U. Krengel: Discussion of Professor’s Ornstein’s paper. (see [45]). Ann. Probab. 1 (1973).
[85]	W. Krieger: On entropy and generators of measure-preserving transformations. Trans. Amer. Math. Soc. 119 (1970), 453-464. Erratum: ibid. 168 (1972), 519. · Zbl 0204.07904 · doi:10.2307/1995407
[86]	W. Krieger: On unique ergodicity. Proc. 6th Berkeley Symp. Math. Stat. Prob., Vol. I. University of California Press, Los Angeles 1972, 327-346. · Zbl 0262.28013
[87]	N. Kryloff, N. Bogoliouboff: La théorie générale de la mesure dans son application à ľétude des systémes dynamiques de la mécanique non linéaire. Ann. Math. 38 (1937), 65-113. · JFM 63.1002.01
[88]	A. G. Kushnirenko: On metric invariants of entropy type. (in Russian). Uspehi Mat. Nauk 22 (1967), 57-65. · Zbl 0169.46101 · doi:10.1070/RM1967v022n05ABEH001225
[89]	A. Leon-Garcia L. D. Davisson, and D. L. Neuhoff: New results on coding of stationary nonergodic sources. IEEE Trans. Inform. Theory IT-25 (1979), 137-144. · Zbl 0406.94007 · doi:10.1109/TIT.1979.1056009
[90]	K. M. Mackenthun, M. B. Pursley: Variable-rate universal block source coding subject to a fidelity criterion. IEEE Trans. Inform. Theory IT-24 (1978), 349 - 360. · Zbl 0384.94009 · doi:10.1109/TIT.1978.1055880
[91]	B. Marcus: Factors and extensions of full shifts. Monatsh. Math. 88 (1979), 239-247. · Zbl 0432.54036 · doi:10.1007/BF01295238
[92]	B. McMillan: The basic theorems of information theory. Ann. Math. Statist. 24 (1953), 196-219. · Zbl 0050.35501 · doi:10.1214/aoms/1177729028
[93]	L. D. Meshalkin: One particular case of isomorphism of Bernoulli schemes. (in Russian). Doklady AN SSR 141 (1959), 41-44. · Zbl 0099.12301
[94]	M. Morse: Symbolic Dynamics. (lecture notes). Institute for Advanced Study, Princeton 1966. · Zbl 0164.22201
[95]	D. L. Neuhoff R. M. Gray.and L. D. Davisson: Fixed-rate universal block source coding with a fidelity criterion. IEEE Trans. Inform. Theory IT-22 (1975), 524-532. · Zbl 0313.94007
[96]	D. L. Neuhoff, P. C. Shields: Fixed-rate universal codes for Markov sources. IEEE Trans. Inform. Theory IT-24 (1978), 360-367. · Zbl 0404.94006 · doi:10.1109/TIT.1978.1055881
[97]	D. L. Neuhoff, P. C. Shields: Channels with almost finite memory. IEEE Trans. Inform. Theory 1T-25 (1979), 440-447. · Zbl 0409.94011 · doi:10.1109/TIT.1979.1056074
[98]	D. L. Neuhoff, P. C. Shields: Indecomposable finite state channels and primitive approximation. IEEE Trans. Inform. Theory IT-28 (1982), 11-18. · Zbl 0483.94011 · doi:10.1109/TIT.1982.1056449
[99]	D. S. Ornstein: Bernoulli shifts with the same entropy are isomorphic. Adv. in Math. 4 (1970), 338-352. · Zbl 0197.33502 · doi:10.1016/0001-8708(70)90029-0
[100]	D. S. Ornstein: Factors of Bernoulli shifts are Bernoulli. Adv. in Math. 5 (1970), 349-364. · Zbl 0227.28015
[101]	D. S. Ornstein: Imbedding Bernoulli shifts in flows. Contributions to Ergodic Theory and Probability. (Lecture Notes in Mathematics 160.) Springer-Verlag, Berlin-Heidelberg-New York 1970, 178-218. · Zbl 0227.28013
[102]	D. S. Ornstein: An application of ergodic theory to probability theory. Ann. Probab. 1 (1973), 43-65. · Zbl 0282.28009 · doi:10.1214/aop/1176997024
[103]	D. S. Ornstein: Ergodic Theory, Randomness, and Dynamical Systems. Yale Univ. Press, New Haven-London 1974. · Zbl 0296.28016
[104]	D. S. Ornstein, B. Weiss: Ergodic theory of amenable group actions. I: The Rohlin lemma. Bull. (New Series) Amer. Math. Soc. 2 (1980), 161-164. · Zbl 0427.28018 · doi:10.1090/S0273-0979-1980-14702-3
[105]	J. C. Oxtoby: Ergodic sets. Bull. Amer. Math. Soc. 58 (1952), 116-136. · Zbl 0046.11504 · doi:10.1090/S0002-9904-1952-09580-X
[106]	W. Parry: Intrinsic Markov chains. Trans. Amer. Math. Soc. 112 (1964), 55 - 66. · Zbl 0127.35301 · doi:10.2307/1994009
[107]	W. Parry: Entropy and Generators in Ergodic Theory. W. A. Benjamin, New York-Amsterdam 1969. · Zbl 0175.34001
[108]	W. Parry: A finitary classification of topological Markov chains and sofic systems. Bull. London Math. Soc. 9 (1977), 86-92. · Zbl 0352.60054 · doi:10.1112/blms/9.1.86
[109]	W. Parry: Endomorphisms of a Lebesgue space III. Israel J. Math. 21 (1975), 167-172. · Zbl 0312.28018 · doi:10.1007/BF02760795
[110]	W. Parry: The information cocycle and \(\epsilon\)-bounded codes. Israel J. Math. 29 (1978), 205-230. · Zbl 0394.94010 · doi:10.1007/BF02762010
[111]	W. Parry: An information obstruction to finite expected coding length. Ergodic Theory. Proceedings, Oberwolfach. (Lecture Notes in Mathematics 729.) Springer-Verlag, Berlin-Heidelberg-New York 1979, 163-168. · Zbl 0414.28016
[112]	W. Parry: Finitary isomorphisms with finite expected code-length. Bull. London Math. Soc. 11 (1979), 170- 176. · Zbl 0425.60029 · doi:10.1112/blms/11.2.170
[113]	W. Parry: Topics in Ergodic Theory. (Cambridge Tracts in Mathematics 75.) Cambridge Univ. Press, Cambridge 1981. · Zbl 0449.28016
[114]	W. Parry, K. Schmidt: A note on cocycles of unitary representations. Proc. Amer. Math. Soc. 55 (1976), 185-190. · Zbl 0329.22008 · doi:10.2307/2041869
[115]	W. Parry, S. Tuncel: On the classification of Markov chains by finite equivalence. Preprint. Warwick Univ., Math. Institute, March 1981. · Zbl 0485.60063
[116]	K. R. Parthasarathy: Probability Measures on Metric Spaces. Academic Press, New York 1967. · Zbl 0153.19101
[117]	M. B. Pursley, L. D. Davisson: Variable-rate coding for nonergodic sources and classes of sources subject to a fìdelity constraint. IEEE Trans. Inform. Theory IT-22 (1976), 324-337. · Zbl 0328.94015 · doi:10.1109/TIT.1976.1055539
[118]	M. B. Pursley, K. M. Mackenthun: Variable-rate coding for classes of sources with generalized alphabets. IEEE Trans. Inform. Theory IT-23 (1977), 592-597. · Zbl 0364.94026 · doi:10.1109/TIT.1977.1055768
[119]	V. A. Rohlin: On basic concepts of measure theory. (in Russian). Mat. Sbornik 67 (1949), 107-150.
[120]	V.A. Rohlin: Selected problems of the metric theory of dynamical systems. (in Russian). Uspehi Mat. Nauk 30 (1949), 57-128.
[121]	V. A. Rohlin: On the decomposition of a dynamical system into transitive components. (in Russian). Mat. Sbornik 67 (1949), 235-249.
[122]	V. A. Rohlin, Ya. G. Sinai: Construction and properties of invariant measurable partitions. (in Russian). Doklady AN SSSR 141 (1961), 1038 - 1041.
[123]	D. J. Rudolph: A characterization of those processes finitarily isomorphic to a Bernoulli shift. Ergodic Theory and Dynamical Systems I. Progress in Mathematics, Vol. 10. Birkhäuser, Boston, Mass. 1981, 1 - 64. · Zbl 0483.28015
[124]	D. J. Sakrison: The rate distortion function of a class of sources. Inform. and Control 15 (1969), 165-195. · Zbl 0185.47703
[125]	C. E. Shannon: A mathematical theory of communication. Bell. System Techn. J. 27 (1948), 379-432, 623-656. · Zbl 1154.94303
[126]	C. E. Shannon: Coding theorems for discrete source with a fidelity criterion. IRE Nat. Conv. Rec, part 4 (1959), 142-163.
[127]	C. E. Shannon: The zero-error capacity of a noisy channel. IRE Trans. 3 (1056), 8 - 32.
[128]	P. C. Shields: The Theory of Bernoulli Shifts. Univ. of Chicago Press, Chicago 1973. · Zbl 0308.28011
[129]	P. C. Shields: Stationary coding of processes. IEEE Trans. Inform. Theory IT-25 (1979), 283-291. · Zbl 0401.94018 · doi:10.1109/TIT.1979.1056037
[130]	P. C. Shields: Almost block independence. Z. Wahrsch. verw. Gebiete 49 (1979), 119-123. · Zbl 0393.60031 · doi:10.1007/BF00534344
[131]	P. C. Shields, D. L. Neuhoff: Block and sliding-block source coding. IEEE Trans. Inform. Theory 1T-23 (1977), 211-215. · Zbl 0383.94018 · doi:10.1109/TIT.1977.1055694
[132]	K. Sigmund: On the prevalence of zero entropy. Israel J. Math. 10 (1971), 281-288. · Zbl 0225.28012 · doi:10.1007/BF02771645
[133]	Ya. G. Sinai: On the notion of entropy of a dynamical system. (in Russian). Doklady AN SSSR 124 (1959), 768-771. · Zbl 0086.10102
[134]	Ya. G. Sinai: On weak isomorphism of transformations with an invariant measure. (in Russian). Mat. Sbornik 63 (1964), 23-42.
[135]	S. Smale: Differentiable dynamical systems. Bull. Amer. Math. Soc. 73 (1967), 747-817. · Zbl 0202.55202 · doi:10.1090/S0002-9904-1967-11798-1
[136]	M. Smorodinsky: Ergodic Theory, Entropy. (Lecture Notes in Mathematics 214.) Springer-Verlag, Berlin- Heidelberg- New York 1971. · Zbl 0213.07502
[137]	M. Smorodinsky: A partition on a Bernoulli shift which is not weak Bernoulli. Math. Systems Theory 5 (1971), 201-203. · Zbl 0226.60066 · doi:10.1007/BF01694176
[138]	Š. Šujan: Generators of an abelian group of invertible measure-preserving transformations. Monatsh. Math. 90 (1980), 68-79. · Zbl 0432.28016 · doi:10.1007/BF01641712
[139]	Š. Šujan: Epsilon-rates, epsilon-quantiles, and group coding theorems for firiitely additîve information sources. Kybernetika 16 (1980), 105-119. · Zbl 0441.94008
[140]	Š. Šujan: Existence of asymptotic rate for asymptotically mean stationary sources with countable alphabets. Trans. 3rd Czechosl.-Soviet-Hung. Seminar on Information Theory. ÚTIA ČSAV, Prague 1980, 201-207. · Zbl 0522.94010
[141]	Š. Šujan: Channels with additive asymptotically mean stationary noise. Kybernetika 17 (1981), 1-15. · Zbl 0455.94008
[142]	Š. Šujan: On the capacity of asymptotically mean stationary channels. Kybernetika 17 (1981), 122-233. · Zbl 0483.94009
[143]	Š. Šujan: Continuity and quantization of channels with infinite alphabets. Kybernetika 17 (1981), 465-478. · Zbl 0479.94012
[144]	Š. Šujan: Block transmissibility and quantization. Probability and Statistical Inference. (W. Grossmann et al., D. Reidel, Dordrecht-Boston-London 1982, 361 - 371.
[145]	Š. Šujan: A local structure of stationary perfectly noiseless codes between stationary nonergodic sources. I: General considerations. Kybernetika 18 (1982), 361-376. · Zbl 0504.94021
[146]	Š. Šujan: A local structure of stationary perfectly noiseless codes between stationary nonergodic sources. II: Applications. Kybernetika 18 (1982), 465-484. · Zbl 0533.94001
[147]	Š. Šujan: Codes in ergodic theory and information: Some examples. Proc. Conf. Ergodic Theory and Related Topics, Akademie-Verlag, Berlin 1982 · Zbl 0514.94007
[148]	Š. Šujan: Finite generators for amenable group actions. · Zbl 0498.28020
[149]	J.-P. Thouvenot: Quelques proprietes des systémes dynamiques qui se decomposent en un produit de deux systémes dont l’un est un schema de Bernoulli. Israel J. Math. 21 (1975), 178-207. · Zbl 0329.28008 · doi:10.1007/BF02760797
[150]	S. Tuncel: Conditional pressure and coding. Israel J. Math. 39 (1981), 101 - 112. · Zbl 0472.28016 · doi:10.1007/BF02762856
[151]	P. Walters: Ergodic Theory. Introductory Lectures. (Lectures Notes in Mathematics 458.) Springer-Verlag, Berlin -Heidelberg-New York 1975. · Zbl 0299.28012
[152]	B. Weiss: The isomorphism problem in ergodic theory. Bull. Amer. Math. Soc. 78 (1972), 668-684. · Zbl 0255.28014 · doi:10.1090/S0002-9904-1972-12979-3
[153]	K. Winkelbauer: On discrete information sources. Trans. 3rd Prague Conf. Inform. Theory etc., NČSAV, Prague 1964, 765-830. · Zbl 0126.35702
[154]	K. Winkelbauer: On the asymptotic rate of nonergodic information sources. Kybernetika 6 (1970), 128-148. · Zbl 0245.94013
[155]	K. Winkelbauer: On the existence of finite generators for invertible measure-preserving transformations. Comment. Math. Univ. Carolinae 18 (1977), 789 - 812. · Zbl 0368.28020
[156]	J. Wolfowitz: Coding Theorems of Information Theory. 2nd Springer-Verlag, New York 1964. · Zbl 0132.39704
[157]	G. M. Zaslavskij: On the isomorphism problem for stationary processes. (in Russian). Teoria veroyatnostej i primen. 9 (1964), 241 - 298. · Zbl 0154.18901
[158]	J. Ziv: Coding of sources with unknown statistics. Part I: Probability of encoding error; Part II: Distortion relative to a fidelity criterion. IEEE Trans. Inform. Theory IT-18 (1972), 384-394. · Zbl 0247.94007
[159]	J. Ziv: Coding theorems for individual sequences. IEEE Trans. Inform. Theory IT-24 (1978), 405-413. · Zbl 0416.94006 · doi:10.1109/TIT.1978.1055911
[160]	J. Ziv, A. Lempel: Compression of individual sequences via variable-rate coding. IEEE Trans. Inform. Theory IT-24 (1978), 530-536. · Zbl 0392.94004 · doi:10.1109/TIT.1978.1055934