zbMATH — the first resource for mathematics

Potential modeling: conditional independence matters. (English) Zbl 1305.62236
Summary: Weights-of-evidence is the special case of logistic regression, if the predictor variables are conditionally independent indicator variables given the target variable. In this case the contrasts of the weights of evidence are identical to the parameters of the corresponding logistic regression model. If the modeling assumption of conditional independence is not satisfied, application of weights-of-evidence corrupts both the predicted conditional probabilities as well as their rank transforms. On the other hand, a logistic regression model including corresponding interaction terms compensates the lack of conditional independence exactly and is optimum. Thus, logistic regression including interaction terms is the canonical generalization of the naïve Bayesian approach assuming conditional independence of all predictor variables given the target variable. Looking at \(2\)-tuples of the conditional probability of an event and its complement, and replacing the logit transform of logistic regression by the isometric log-ratio transform of compositional statistics, leads to similar compositional regression models which in turn yield very similar numerical results. Artificial neural nets generalize logistic regression by way of nesting regression-like models. Thus they are generally capable to model more involved relationships between predictor variables and the target variable. They are controlled by a lot of parameters, their ultimate characteristic being the topology of the net. Artificial neural nets do not generally provide a measure of confidence in their parameters, in particular they do not feature the concept of statistical significance. Applying the methods mentioned above to a simple example with fabricated data evidences the impact on the predictions and their rank transforms, if the assumption of conditional independence of the predictor variables given the target variable is not satisfied and not taken into account by interaction terms.

62J02 General nonlinear regression
compositions; R
Full Text: DOI
[1] Agterberg, F.P., Bonham-Carter, G.F., Wright, D.F.: Statistical pattern integration for mineral exploration. In: Gaál, G., Merriam, D.F. (eds.) Computer Applications in Resource Estimation Prediction and Assessment for Metals and Petroleum, pp. 1–21. Pergamon Press, Oxford–New York (1990)
[2] Aitchison, J.: The Statistical Analysis of Compositional Data. Chapman & Hall, reprinted in 2003 with additional material by The Blackburn Press (1986)
[3] Aitchison, J.: The statistical analysis of compositional data (with discussion). J. R. Stat. Soc. Ser. B (Statistical Methodology) 44, 139–177 (1982) · Zbl 0491.62017
[4] Bandemer, H., Näther, W.: Fuzzy data analysis. Springer, Berlin (1992) · Zbl 0776.94021
[5] Billheimer, D., Guttorp, P., Fagan, W.F.: Statistical interpretation of species composition. J. Am. Stat. Assoc. 96, 1205–1214 (2001) · Zbl 1073.62573 · doi:10.1198/016214501753381850
[6] Bishop, C.M.: Patter recognition and machine learning. Springer, Berlin (2006) · Zbl 1107.68072
[7] Bonham-Carter, G.F., Agterberg, F.P.: Application of a microcomputer based geographic information system to mineral-potential mapping. In: Hanley, J.T., Merriam, D.F. (eds.) Microcomputer-based Applications in Geology, II, Petroleum, pp. 49–74. Pergamon Press, New York (1990)
[8] Chilès, J.-P., Delfiner, P.: Geostatistics-Modeling Spatial Uncertainty. Wiley, New York (1999) · Zbl 0922.62098
[9] Chung, C.-J., Fabbri, A.G.: Modelling target maps of future gold occurrences with combination of categorical and continuous conditionally dependent supporting patterns. In: Proceedings of the 12th SGA Biennial Meeting, pp. 476–479. Uppsala, Sweden, 12–15 August 2013
[10] Fabbri, A.G., Poli, S., Patera, A., Cavallin, A., Chung, C.-J.: Estimation of information loss when masking conditional dependence and categorizing continuous data, Further experiments on a database for spatial prediction modelling in Northern Italy. In: 15th Annual Conference of the International Association for Mathematical Geosciences, Madrid, 2–6 September (2013)
[11] Good, I.J.: The Estimation Of Probabilities: An Essay on Modern Bayesian Methods. Research Monograph No. 30. The MIT Press, Cambridge, MA, USA (1968) · Zbl 0168.39603
[12] Good, I.J.: Probability and the Weighing of Evidence. Griffin, London (1950) · Zbl 0036.08402
[13] Hand, D.J., Yu, K.: Idiot’s Bayes - not so stupid after all? Int. Stat. Rev. 69, 385–398 (2001) · Zbl 1213.62010
[14] Harris, D.V., Zurcher, L., Stanley, M., Marlow, J., Pan, G.: A comparative analysis of favorability mappings by weights of evidence, probabilistic neural networks, discriminant analysis, and logistic regression. Nat. Resour. Res. 12, 241–255 (2003) · doi:10.1023/B:NARR.0000007804.27450.e8
[15] Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Berlin (2001) · Zbl 0973.62007
[16] Hosmer, D.W., Lemeshow, S.: Applied Logistic Regression, 2nd edn. Wiley, New York (2000) · Zbl 0967.62045
[17] Hronsky, J.M.A., Groves, D.I.: Science of targeting: definition, strategies, targeting and performance measurement. Aust. J. Earth. Sci. 55, 3–12 (2008) · doi:10.1080/08120090701581356
[18] Moguerza, J.M., Muñoz, A.: Support vector machines with applications. Stat. Sci. 21, 322–336 (2006) · Zbl 1246.68185 · doi:10.1214/088342306000000493
[19] Müller, P., Rios Insua, D.: Issues in Bayesian analysis of neural network models. Neural Comput. 10, 740–770 (1998)
[20] Pawlowsky-Glahn, V., Buccianti, A.: Compositional Data Analysis: Theory and Applications. Wiley, New York (2011) · Zbl 1103.62111
[21] Pawlowsky-Glahn, V., Egozcue, J.-J.: Compositional data and their analysis–an introduction. Geol. Soc. Lond. Spec. Publ. 264, 1–10 (2006). doi: 10.1144/GSL.SP.2006.264.01.01 · Zbl 1156.86308 · doi:10.1144/GSL.SP.2006.264.01.01
[22] Pearson, K.: Mathematical contributions to the theory of evolution. On a form of spurious correlations which may arise when indices are used in the measurement of organs. Proc. R. Soc. Lond. 60, 489–498 (1897) · JFM 28.0209.02 · doi:10.1098/rspl.1896.0076
[23] Porwal, A., Carranza, E.J.M., Hale, M.: Knowledge-driven and data-driven fuzzy models for predictive mineral potential mapping. Nat. Resour. Res. 12, 1–25 (2003) · doi:10.1023/A:1022693220894
[24] R Development Core Team: R–A language and environment for statistical computing. R Foundation for Statistical Computing, http://www.R-project.org/ , Vienna, Austria (2013)
[25] Russell, S., Norvig, P.: Artificial Intelligence, a Modern Approach, 2nd edn. Prentice Hall, Englewood Cliffs (2003) · Zbl 0835.68093
[26] Schaeben, H.: A mathematical view of weights-of-evidence, conditional independence, and logistic regression in terms of markov random fields. Math. Geosci. (2013). doi: 10.1007/s11004-013-9513-y · Zbl 1323.60073
[27] Schaeben, H., van den Boogaart, K.G.: Comment on ”A conditional dependence adjusted weights of evidence model” by Minfeng Deng in Natural Resources Research 18(2009), 249–258. Nat. Resour. Res. 20, 401–406 (2011) · doi:10.1007/s11053-011-9146-0
[28] Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002) · Zbl 1019.68094
[29] Skabar, A.: Modeling the spatial distribution of mineral deposits using neural networks. Nat. Resour. Model. 20, 435–450 (2007) · Zbl 1159.86307 · doi:10.1111/j.1939-7445.2007.tb00215.x
[30] Smola, A.J., Vishwanathan, S.V.N.: Introduction to Machine Learning. Cambridge University Press, Cambridge (2008)
[31] Sutton, C., McCallum, A.: An introduction to conditional random fields for relational learning. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning, pp. 93–127. MIT Press, Cambridge (2007)
[32] van den Boogaart, K.G., Tolosana-Delgado, R.: Analyzing Compositional Data with R. Springer, Berlin (2013) · Zbl 1276.62011
[33] Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Berlin (2000) · Zbl 0934.62009
[34] Zhang, D., Agterberg, F.P., Cheng, Q., Zuo, R.: A comparison of modified fuzzy weights of evidence, fuzzy weights of evidence, and logistic regression for mapping mineral prospectivity. Math. Geosci. (2013). doi: 10.1007/s11004-013-9496-8 · Zbl 1323.86037
[35] Zhang, K., Peters, J., Janzing, D., Schölkopf, B.: Kernel-based conditional independence test and application in causal discovery. In: Cozman, F.G., Pfeffer, A. (eds.) Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), pp. 804–813. AUAI Press, Corvallis (2011)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.