×

A mathematical view of weights-of-evidence, conditional independence, and logistic regression in terms of Markov random fields. (English) Zbl 1323.60073

Summary: New light is shed on mathematical methods of potential modeling from the point of view of Markov random fields. In particular, weights-of-evidence and logistic regression models are discussed in terms of graphical models possessing Markov properties, where the notion of conditional independence is essential, and will be related to log-linear models. While weights-of-evidence with respect to indicator predictor variables and logistic regression with unrestricted predictor variables model conditional probabilities of an indicator random target variable, the subject of log-linear models is the joint probability of random variables. The relationship to log-linear models leads to a likelihood ratio test of conditional independence, rendering an omnibus test of conditional independence restricted by a normality assumption obsolete. Moreover, it reveals a hierarchy of methods comprising weights-of-evidence, logistic regression without interaction terms, and logistic regression including interaction terms, where each former method is a special case of the consecutive latter method. The assumptions of conditional independence of all predictor variables given the target variable lead to logistic regression without interaction terms. Violations of conditional independence are compensated exactly by corresponding interaction terms, no cumbersome approximate corrections are needed. Thus, including interaction terms into logistic regression models is an appropriate means to account for lacking conditional independence. Logistic regression exempts from the burden to worry about lack of conditional independence. Eventually, the relationship to log-linear models renders logistic regression with indicator predictor variables optimum for discrete predictor variables. Weights-of-evidence applies for indicator predictor variables only, logistic regression applies without restrictions of the type of predictor variables and approximates the proper distribution in the general case.

MSC:

60G60 Random fields
62P12 Applications of statistics to environmental and related topics
62M40 Random fields; image analysis

Software:

MIM
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Agterberg, FP; Bonham-Carter, GF; Wright, DF; Gaál, G. (ed.); Merriam, DF (ed.), Statistical pattern integration for mineral exploration, 1-21 (1990), Oxford · doi:10.1016/B978-0-08-037245-7.50006-8
[2] Agterberg FP, Bonham-Carter GF, Wright DF, Cheng Q (1989) Weights of evidence and weighted logistic regression for mineral potential mapping: In: Davis JC, Herzfeld UC (eds) Computers in geology: 25 years of progress. Oxford University Press, New York, pp 13-32
[3] Aitchison J (1986) The statistical analysis of compositional data. Chapman & Hall, New York (reprinted in 2003 with additional material by The Blackburn Press) · Zbl 0688.62004
[4] Allard D, Comunian A, Renard P (2012) Probability aggregation methods in geoscience. Math Geosci 44:545-581 · Zbl 1256.86006
[5] Berkson J (1944) Application of the logistic function to bio-assay. J Am Stat Assoc 39:357-365
[6] Bishop CM (2006) Patter recognition and machine learning. Springer, Berlin · Zbl 1247.86011
[7] Bonham-Carter GF, Agterberg FP (1990) Application of a microcomputer based geographic information system to mineral-potential mapping. In: Hanley JT, Merriam DF (eds) Microcomputer-based applications in geology, II. Petroleum. Pergamon Press, New York, pp 49-74
[8] Cheng Q (2008) Non-linear theory and power-law models for information integration and mineral resources quantitative assessments. Math Geosci 40:503-532 · Zbl 1153.86327
[9] Chung C-J, Fabbri AG (2013) Modeling target maps of future gold occurrences with combination of categorical and continuous conditionally-dependent supporting patterns. In: Proceedings of 12th SGA Biennial Meeting, Uppsala, Sweden, vol. 2. Aug 12-15, pp 476-479 · Zbl 1246.68185
[10] Cox DP, Singer DA (eds) (1986) Mineral deposit models: US Geological Survey Bulletin 1693, p 379
[11] Cramer JS (2002) The origins of logistic regression: Tinbergen Institute Discussion Paper TI 2002-119/4
[12] Deng M (2009) A conditional dependence adjusted weights of evidence model. Nat Resour Res 18:249-258 · doi:10.1007/s11053-009-9101-5
[13] Edwards D (2000) Introduction to graphical modelling, 2nd edn. Springer, Berlin · Zbl 0952.62003
[14] Firth D (1993) Bias reduction of maximum likelihood estimates. Biometrika 80:27-38 · Zbl 0769.62021 · doi:10.1093/biomet/80.1.27
[15] Good IJ (1950) Probability and the weighing of evidence. Griffin, London · Zbl 0036.08402
[16] Good IJ (1960) Weight of evidence, corroboration, explanatory power, information and the utility of experiments. J R Stat Soc B 22:319-331 · Zbl 0122.36803
[17] Good IJ (1968) The estimation of probabilities: an essay on modern Bayesian methods. Research monograph no. 30. The MIT Press, Cambridge
[18] Good IJ (1984) C197. The best explicatum for weight of evidence. J Stat Comput Simul 19:294-299
[19] Good IJ (1989) C312. Yet another argument for the explicatum of weight of evidence. J Stat Comput Simul 31:58-59
[20] Good IJ (2003) The accumulation of imprecise weights of evidence. http://www.sipta.org/isipta03/jack.pdf
[21] Good IJ, Osteyee DB (1974) Information, weight of evidence: the singularity between probability measures and signal detection. Springer, Berlin · Zbl 0298.94001
[22] Hand DJ, Yu K (2001) Idiot’s Bayes—not so stupid after all? Int Stat Rev 69:385-398 · Zbl 1213.62010
[23] Hammerslay JM, Clifford PE (1971) Markov fields on random graphs and lattices (unpublished manuscript)
[24] Harary F (1994) Graph theory. Addison-Wesley, Reading · Zbl 0182.57702
[25] Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning, Springer, Berlin · Zbl 0973.62007
[26] Højsgaard S, Edwards D, Lauritzen S (2012) Graphical models with R. Springer, Berlin · Zbl 1286.62005
[27] Hosmer DW, Lemeshow S (2000) Applied logistic regression, 2nd edn. Wiley, New York · Zbl 0967.62045
[28] Journel AG (2002) Combining knowledge from diverse sources: an alternative to traditional data independence hypotheses. Math Geol 34:573-596 · Zbl 1032.86005 · doi:10.1023/A:1016047012594
[29] Krishnan S (2008) The \[\tau\] τ-model for data redundancy and information combination in Earth sciences: theory and application. Math Geol 40:705-727 · Zbl 1179.86004
[30] Krishnan S, Boucher A, Journel AG (2005) Evaluating information redundancy through the \[\tau\] τ-model. In: Leuangthong O, Deutsch CV (eds) Geostatistics Banff 2004. Springer, Berlin, pp 1037-1046 · Zbl 1366.62019
[31] Lauritzen SL, (1996) Graphical models. Clarendon Press, Oxford · Zbl 0907.62001
[32] McCuaig TC, Beresford S, Hronsky J (2010) Translating the mineral systems approach into an effective exploration targeting system. Ore Geol Rev 38:128-138 · doi:10.1016/j.oregeorev.2010.05.008
[33] McCullagh P, Nelder J (1989) Generalized linear models, 2nd edn. Chapman and Hall/CRC, New York · Zbl 0588.62104 · doi:10.1007/978-1-4899-3242-6
[34] Minsky M, Selfridge OG (1961) Learning in random nets. In: Cherry C (ed) Information theory: Fourth London Symposium, Butterworths, London, pp 335-347
[35] Moguerza JM, Muñoz A (2006) Support vector machines with applications. Stat Sci 21:322-336 · Zbl 1246.68185 · doi:10.1214/088342306000000493
[36] Müller P, Rios Insua D (1998) Issues in Bayesian analysis of neural network models. Neural Comput 10:740-770 · doi:10.1162/089976698300017737
[37] Nelder J, Wedderburn R (1972) Generalized linear models. J R Stat Soc Ser A 135:370-384 · doi:10.2307/2344614
[38] Pearl J (1988) Probabilistic reasoning in intelligent systems. Morgan Kaufmann, San Mateo · Zbl 0746.68089
[39] Pearl J, Paz A (1987) Graphoids: a graph based logic for reasoning about relevancy relations. In: Boulay, BD, Hogg D, Steel L (eds) Advances in artificial intelligence II. North-Holland, Amsterdam, pp 357-363
[40] Pierce CS (1878) The Probability of induction: popular science monthly (republished. in Newman JR (ed) The world of mathematics 2. Simon and Schuster, New York, pp 1341-1354)
[41] Polyakova EI, Journel AG (2007) The \[\nu\] ν-expression for probabilistic data integration. Math Geol 39:715-733 · Zbl 1141.86307 · doi:10.1007/s11004-007-9117-5
[42] Reed LJ, Berkson J (1929) The application of the logistic function to experimental data. J Phys Chem 33:760-779 · doi:10.1021/j150299a014
[43] Russell S, Norvig P (2003) Artificial intelligence. A modern approach, 2nd edn. Prentice Hall, New York · Zbl 0835.68093
[44] Schaeben H (2011) Comparison of mathematical methods of potential modeling. Math Geosci 44:101-129. doi:10.1007/s11004-011-9373-2 · Zbl 1247.86011 · doi:10.1007/s11004-011-9373-2
[45] Schaeben H, van den Boogaart KG (2011) Comment on “A conditional dependence adjusted weights of evidence model” by Minfeng Deng in Natural Resources Research 18(2009), 249-258. Nat Resour Res 20:401-406
[46] Singer DA, Menzie WD (2010) Quantitative mineral resource assessments, an integrated approach. Oxford University Press, Oxford
[47] Skabar A (2007) Modeling the spatial distribution of mineral deposits using neural networks. Nat Resour Model 20:435-450 · Zbl 1159.86307 · doi:10.1111/j.1939-7445.2007.tb00215.x
[48] Sutton C, McCallum A (2007) An introduction to conditional random fields for relational learning. In: Getoor L, Taskar B (eds) Introduction to statistical relational learning. MIT Press, Cambridge, pp 93-127
[49] Zhang D, Cheng Q, Agterberg FP (2012) Weights of evidence method and weighted logistic regression model. Abstracts, 34th International Geological Congress, p 1789
[50] Zhang S, Cheng Q (2012) A modified weights of evidence model for mineral potential mapping. Abstracts, 34th International Geological Congress, p 2008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.