zbMATH — the first resource for mathematics

Inconsistency indices for pairwise comparison matrices: a numerical study. (English) Zbl 1286.90076
Summary: Evaluating the level of inconsistency of pairwise comparisons is often a crucial step in multi criteria decision analysis. Several inconsistency indices have been proposed in the literature to estimate the deviation of expert’s judgments from a situation of full consistency. This paper surveys and analyzes ten indices from the numerical point of view. Specifically, we investigate degrees of agreement between them to check how similar they are. Results show a wide range of behaviors, ranging from very strong to very weak degrees of agreement.

90B50 Management decision making, including multiple objectives
Full Text: DOI
[1] Aguaròn, J., & Moreno-Jimènez, J. M. (2003). The geometric consistency index: approximated threshold. European Journal of Operational Research, 147(1), 137–145. · Zbl 1060.90657
[2] Anholcer, V., Babiy, V., Bózoki, S., & Koczkodaj, W. W. (2011). A simplified implementation of the least squares solution for pairwise comparisons matrices. Central European Journal of Operations Research, 19(4), 439–444. · Zbl 1245.90039
[3] Barzilai, J. (1998). Consistency measures for pairwise comparison matrices. Journal of Multi-Criteria Decision Analysis, 7(3), 123–132. · Zbl 0910.90001
[4] Bózoki, S. (2008). Solution of the least squares method problem of pairwise comparison matrices. Central European Journal of Operations Research, 16(4), 345–358. · Zbl 1173.90588
[5] Brunelli, M. (2011). A note on the article ”Inconsistency of pair-wise comparison matrix with fuzzy elements based on geometric mean” [Fuzzy Sets and Systems 161 (2010) 1604–1613]. Fuzzy Sets and Systems, 161(1), 1604–1613. 2010, Fuzzy Sets and Systems, 176, 76–78. · Zbl 1186.90145
[6] Brunelli, M., Critch, A., & Fedrizzi, M. (2013). On the proportionality between some consistency indices in the AHP. Applied Mathematics and Computation. doi: 10.1016/j.amc.2013.01.036 . · Zbl 1288.91043
[7] Cavallo, B., & D’Apuzzo, L. (2009). A general unified framework for pairwise comparison matrices in multicriterial methods. International Journal of Intelligent Systems, 24(4), 377–398. · Zbl 1163.68336
[8] Cavallo, B., & D’Apuzzo, L. (2010). Characterizations of consistent pairwise comparison matrices over abelian linearly ordered groups. International Journal of Intelligent Systems, 25(10), 1035–1059. · Zbl 1202.90156
[9] Choo, E., & Wedley, W. (2004). A common framework for deriving preference values from pairwise comparison matrices. Computers & Operations Research, 31(6), 893–908. · Zbl 1043.62063
[10] Chu, A. T. W., Kalaba, R. E., & Springarn, K. (1979). A comparison of two methods for determining the weights of belonging to fuzzy sets. Journal of Optimization Theory and Applications, 27(4), 531–538. · Zbl 0377.94002
[11] Cohen, J. (1968). Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220.
[12] Crawford, G., & Williams, C. (1985). A note on the analysis of subjective judgement matrices. Journal of Mathematical Psychology, 29(4), 25–40. · Zbl 0585.62183
[13] Duszak, Z., & Koczkodaj, W. W. (1994). Generalization of a new definition of consistency for pairwise comparisons. Information Processing Letters, 52(5), 273–276. · Zbl 0815.68084
[14] Fedrizzi, M., & Brunelli, M. (2009). Fair consistency evaluation for reciprocal relations and in group decision making. New Mathematics and Natural Computation, 5(2), 407–420. · Zbl 1173.91336
[15] Golden, B. L., & Wang, Q. (1989). An alternate measure of consistency. In B. L. Golden, E. A. Wasil, & P. T. Harker (Eds.), The analytic hierarchy process, applications and studies (pp. 68–81). Berlin: Springer.
[16] Karelaia, N., & Hogarth, R. M. (2008). Determinants of linear judgment: a meta-analysis of lens studies. Psychological Bulletin, 134(3), 404–426.
[17] Koczkodaj, W. W. (1993). A new definition of consistency of pairwise comparisons. Mathematical and Computer Modelling, 18(7), 79–84. · Zbl 0804.92029
[18] Linares, P. (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. European Journal of Operational Research, 193(2), 492–498. · Zbl 1180.90155
[19] Miller, G. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63(2), 81–97.
[20] Osei-Bryson, K.-M. (2006). An action learning approach for assessing the consistency of pairwise comparison data. European Journal of Operational Research, 174(1), 234–244. · Zbl 1116.90057
[21] Pelàez, J. I., & Lamata, M. T. (2003). A new measure of consistency for positive reciprocal matrices. Computers & Mathematics with Applications, 46(12), 1839–1845. · Zbl 1121.91334
[22] Phelps, R. H., & Shanteau, J. (1978). Livestock judges: how much information can an expert use? Organizational Behavior and Human Performance, 21(2), 209–219.
[23] Ramík, J., & Korviny, P. (2010). Inconsistency of pair-wise comparison matrix with fuzzy elements based on the geometric mean. Fuzzy Sets and Systems, 161(1), 1604–1613. · Zbl 1186.90145
[24] Ramík, J., & Perzina, R. (2010). A method for solving fuzzy multicriteria decision problems with dependent criteria. Fuzzy Optimization and Decision Making, 9(2), 123–141. · Zbl 1187.90339
[25] Saaty, T. L. (1974). Measuring the fuzziness of sets. Journal of Cybernetics, 4(4), 53–61. · Zbl 0319.02060
[26] Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281. · Zbl 0372.62084
[27] Saaty, T. L. (1980). The analytical hierarchy process. New York: McGraw-Hill. · Zbl 0587.90002
[28] Saaty, T. L. (1994). Highlights and critical points in the theory and application of the analytic hierarchy process. European Journal of Operational Research, 74(3), 426–447. · Zbl 0925.90233
[29] Salo, A. A. (1993). Inconsistency analysis by approximately specified priorities. Mathematical and Computer Modelling, 17(4–5), 123–133. · Zbl 0768.90002
[30] Salo, A. A., & Hämäläinen, R. P. (1995). Preference programming through approximate ratio comparisons. European Journal of Operational Research, 82(3), 458–475. · Zbl 0909.90006
[31] Shanteau, J. (1989). Psychological characteristics and strategies of expert decision makers. In B. Rohrmann, L. R. Beach, C. Vleck, & S. R. Watson (Eds.), Advances in decision research (pp. 203–215). Amsterdam: North-Holland.
[32] Shanteau, J., Weiss, D. J., Thomas, R. P., & Pounds, J. (2003). How can you tell if someone is an expert? Performance-based assessment of expertise. In S. L. Schneider & J. Shanteau (Eds.), Emerging perspectives on judgment and decision research (pp. 620–642). Cambridge: Cambridge University Press.
[33] Shiraishi, S., & Obata, T. (2002). On a maximization problem arising from a positive reciprocal matrix in the AHP. Bulletin of Informatics and Cybernetics, 34(2), 91–96. · Zbl 1270.91019
[34] Shiraishi, S., Obata, T., & Daigo, M. (1998). Properties of a positive reciprocal matrix and their application to AHP. Journal of the Operations Research Society of Japan, 41(3), 404–414. · Zbl 1003.15019
[35] Shiraishi, S., Obata, T., Daigo, M., & Nakajima, N. (1999). Assessment for an incomplete matrix and improvement of the inconsistent comparison: computational experiments. In Proceedings of ISAHP 1999, Kobe, Japan.
[36] Slovic, P. (1969). Analyzing the expert judge: a descriptive study of a stockbroker’s decision processes. Journal of Applied Psychology, 53(4), 255–263.
[37] Snedecor, G. W., & Cochran, W. C. (1980). Statistical methods. Ames: Iowa State University Press. · Zbl 0727.62003
[38] Spearman, C. E. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101.
[39] Stein, W. E., & Mizzi, P. J. (2007). The harmonic consistency index for the analytic hierarchy process. European Journal of Operational Research, 177(1), 488–497. · Zbl 1111.90057
[40] Thurstone, L. L. (1927). A law of comparative judgement. Psychological Review, 34(4), 273–386.
[41] Weiss, D. J., & Shanteau, J. (2003). Empirical assessment of expertise. Human Factors, 45(1), 104–114.
[42] Weiss, D. J., Shanteau, J., & Harries, P. (2006). People who judge people. Journal of Behavioral Decision Making, 19(5), 441–454.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.