Evaluation of software development projects using a fuzzy multi-criteria decision approach. (English) Zbl 1147.68452

Summary: Software development is an inherently uncertain activity. To deal with the uncertainty and vagueness from humans’ subjective perception and experience in decision process, this paper presents an evaluation model based on the fuzzy multi-criteria decision-making (MCDM) method for measuring the performance of software development projects. In an MCDM problem, a decision maker (DM) has to choose the best alternative that satisfies the evaluation criteria among a set of candidate solutions. It is generally hard to find an alternative that meets all the criteria simultaneously, so a good compromise solution is preferred.
This problem may become more complex when multiple DMs are involved, each having not a common perception on the alternatives. Recently, a compromise ranking method (known as the VIKOR method) has been proposed to identify such compromise solutions, by providing a maximum group utility for the majority and a minimum of an individual regret for the opponent. In its actual setting, the method treats exact values for the assessment of the alternatives, which can be quite restrictive with unquantifiable criteria. This will be true especially if the evaluation is made by means of linguistic terms. For this reason we extend the VIKOR method so as to process such data and to provide a more comprehensive evaluation in a fuzzy environment. To demonstrate the potential of the methodology, the proposed extension is used for measuring the performance of enterprise resource planning (ERP) software products.


68N99 Theory of software
Full Text: DOI


[1] Baki, B.; Çakar, K., Determining the ERP package-selecting criteria: the case of Turkish manufacturing companies, Bus. Process Manage. J., 11, 75-86 (2005)
[2] Bojadziev, G.; Bojadziev, M., Fuzzy Logic for Business, Finance, and Management, Advances in Fuzzy Systems, vol. 12 (1997), World Scientific Publication · Zbl 1050.91500
[3] J. Bresnahan, Mission possible, CIO Magazine, October 15, 1996.; J. Bresnahan, Mission possible, CIO Magazine, October 15, 1996.
[4] Büyüközkan, G.; Kahraman, C.; Ruan, D., A fuzzy multi-criteria decision approach for software development strategy selection, Int. J. Gen. Syst., 33, 259-280 (2004) · Zbl 1093.68567
[5] Carney, D. J.; Wallnau, K. C., A basis for evaluation of commercial software, Inf. Software Technol., 40, 851-860 (1998)
[6] Cha, Y.; Jung, M., Satisfaction assessment of multi-objective schedules using neural fuzzy methodology, Int. J. Prod. Res., 41, 1831-1849 (2003) · Zbl 1059.90081
[7] Chen, C.-T., Extensions of the TOPSIS for group decision-making under fuzzy environment, Fuzzy Sets Syst., 114, 1-9 (2000) · Zbl 0963.91030
[8] Cheng, C. H.; Lin, Y., Evaluating the best main battle tank using fuzzy decision theory with linguistic criteria evaluation, Eur. J. Oper. Res., 142, 174-186 (2002) · Zbl 1081.90584
[9] Chiclana, F.; Herrera, F.; Herrera-Viedman, E., Integrating three representation models in fuzzy multipurpose decision making based on fuzzy preference relations, Fuzzy Sets Syst., 97, 33-48 (1998) · Zbl 0932.91012
[10] Fortemps, P.; Roubens, M., Ranking and defuzzification methods based on area compensation, Fuzzy Sets Syst., 82, 319-330 (1996) · Zbl 0886.94025
[11] Gediga, G.; Hamborg, K. C.; Düntsch, I., Evaluation of software systems, Encyclopedia Comput. Sci. Technol., 45, 127-153 (2002)
[12] Herrera, F.; Herrera-Viedma, E.; Chiclana, F., Multiperson decision-making based on multiplicative preference relations, Eur. J. Oper. Res., 129, 372-385 (2001) · Zbl 0980.90041
[13] Jiang, J. J.; Klein, G.; Hwang, H.-G.; Huang, J.; Hung, S.-Y., An exploration of the relationship between software development process maturity and project performance, Inf. Manage., 41, 279-288 (2004)
[14] Kahraman, C.; Büyüközkan, G.; Ruan, D., Measuring software development value using fuzzy logic, (Ruan, D.; Zeng, X., Intelligent Sensory Evaluation-Methodologies and Applications (2004), Springer), 285-308
[15] Kaufmann, A.; Gupta, M. M., Fuzzy Mathematical Models in Engineering and Management Science (1988), North-Holland: North-Holland Amsterdam · Zbl 0683.90024
[16] Kaufmann, A.; Gupta, M. M., Introduction to Fuzzy Arithmetic Theory and Applications (1991), Van Nostrand Reinhold: Van Nostrand Reinhold New York · Zbl 0754.26012
[17] Kelly, D. P.; Oshana, R. S., Improving software quality using statistical testing techniques, Inf. Software Technol., 42, 801-807 (2000)
[18] Lai, V. S.; Wong, B. K.; Cheung, W., Group decision making in a multiple criteria environment: a case using the AHP in software selection, Eur. J. Oper. Res., 137, 134-144 (2002) · Zbl 1003.90509
[19] Lee, J. W.; Kim, S. H., Using analytic network process and goal programming for interdependent information system project selection, Comput. Oper. Res., 27, 367-382 (2000) · Zbl 0973.90015
[20] Ngai, E. W.T.; Chan, E. W.C., Evaluation of knowledge management tools using AHP, Expert Syst. Appl., 29, 889-899 (2005)
[21] Opricovic, S., Multicriteria Optimization of Civil Engineering Systems (1998), Faculty of Civil Engineering: Faculty of Civil Engineering Belgrade
[22] Opricovic, S.; Tzeng, G. H., Fuzzy multicriteria model for postearthquake land-use planning, Nat. Hazard. Rev., 4, 59-64 (2003)
[23] Opricovic, S.; Tzeng, G. H., Defuzzification within a multicriteria decision model, Int. J. Uncertain. Fuzz., 11, 635-652 (2003) · Zbl 1072.68619
[24] Opricovic, S.; Tzeng, G. H., Compromise solution by MCDM methods: a comparative analysis of VIKOR and TOPSIS, Eur. J. Oper. Res., 156, 445-455 (2004) · Zbl 1056.90090
[25] Osmundson, J. S.; Michael, J. B.; Machniak, M. J.; Grossman, M. A., Quality management metrics for software development, Inf. Manage., 40, 799-812 (2003)
[26] Stamelos, I.; Vlahavas, I.; Refanidis, I.; Tsoukias, A., Knowledge based evaluation of software systems: a case study, Inf. Software Technol., 42, 333-345 (2000)
[27] Stamelos, I.; Tsoukiàs, A., Software evaluation problem situations, Eur. J. Oper. Res., 145, 273-286 (2003) · Zbl 1011.90507
[28] Tzeng, G. H.; Lin, C. W.; Opricovic, S., Multi-criteria analysis of alternative-fuel buses for public transportation, Energy Policy, 33, 1373-1383 (2005)
[29] Verville, J.; Halingten, A., A six-stage model of the buying process for ERP software, Ind. Market. Manage., 32, 585-594 (2003)
[30] Wang, X.; Kerre, E. E., Reasonable properties for the ordering of fuzzy quantities (I), Fuzzy Sets Syst., 118, 375-385 (2001) · Zbl 0971.03054
[31] Wei, C.-C.; Wang, M-J. J., A comprehensive framework for selecting an ERP system, Int. J. Project Manage., 22, 161-169 (2004)
[32] Wei, C.-C.; Chien, C.-F.; Wang, M-J. J., An AHP-based approach to ERP system selection, Int. J. Prod. Econ., 96, 47-62 (2005)
[33] Yu, P. L., A class of solutions for group decision problems, Manage. Sci., 19, 936-946 (1973) · Zbl 0264.90008
[34] Zadeh, L. A., Fuzzy sets, Inf. Control, 8, 338-353 (1965) · Zbl 0139.24606
[35] Zadeh, L. A., The concept of a linguistic variable and its applications to approximate reasoning, Part I, Inf. Sci., 8, 199-249 (1975), Part II, 8, 301-357; Part III, 9, 43-80 · Zbl 0397.68071
[36] Zeleny, M., Multiple Criteria Decision Making (1982), Mc-Graw-Hill: Mc-Graw-Hill New York · Zbl 0588.90019
[37] Zhang, X.; Pham, H., Software field failure rate prediction before software deployment, J. Syst. Software, 79, 291-300 (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.