Hinton, Geoffrey E. Training products of experts by minimizing contrastive divergence. (English) Zbl 1010.68111 Neural Comput. 14, No. 8, 1771-1800 (2002). Summary: It is possible to combine multiple latent-variable models of the same data by multiplying their probability distributions together and then renormalizing. This way of combining individual ‘expert’ models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert, because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data. A Product of Experts (PoE) is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary. Training a PoE by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule. Fortunately, a PoE can be trained using a different objective function called “contrastive divergence” whose derivatives with regard to the parameters can be approximated accurately and efficiently. Examples are presented of contrastive divergence learning using several types of expert on several types of data. Cited in 1 ReviewCited in 188 Documents MSC: 68T05 Learning and adaptive systems in artificial intelligence 68T35 Theory of languages and software systems (knowledge-based systems, expert systems, etc.) for artificial intelligence Keywords:product of experts × Cite Format Result Cite Review PDF Full Text: DOI References: [1] Berger A., Computational Linguistics 22 pp 39– (1996) [2] DOI: 10.1109/TPAMI.1984.4767596 · Zbl 0573.62030 · doi:10.1109/TPAMI.1984.4767596 [3] DOI: 10.1214/ss/1177013825 · doi:10.1214/ss/1177013825 [4] DOI: 10.1162/089976698300017232 · doi:10.1162/089976698300017232 [5] DOI: 10.1126/science.7761831 · doi:10.1126/science.7761831 [6] DOI: 10.1016/0004-3702(92)90065-6 · Zbl 0761.68081 · doi:10.1016/0004-3702(92)90065-6 [7] DOI: 10.1162/neco.1996.8.5.895 · doi:10.1162/neco.1996.8.5.895 [8] DOI: 10.1109/34.506410 · doi:10.1109/34.506410 [9] DOI: 10.1162/089976600300015385 · doi:10.1162/089976600300015385 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.