NHPP software reliability and cost models with testing coverage. (English) Zbl 1011.90018

Summary: This paper proposes a software reliability model that incorporates testing coverage information. Testing coverage is very important for both software developers and customers of software products. For developers, testing coverage information helps them to evaluate how much effort has been spent and how much more is needed. For customers, this information estimates the confidence of using the software product. Although research has been conducted and software reliability models have been developed, some practical issues have not been addressed. Testing coverage is one of these issues. The model is developed based on a NonHomogeneous Poisson Process (NHPP) and can be used to estimate and predict the reliability of software products quantitatively. We examine the goodness-of-fit of this proposed model and present the results using several sets of software testing data. Comparisons of this model and other existing NHPP models are made. We find that the new model can provide a significant improved goodness-of-fit and estimation power. A software cost model incorporating testing coverage is also developed. Besides some traditional cost items such as testing cost and error removal cost, risk cost due to potential faults in the uncovered code is also included associated with the number of demands from customers. Optimal release policies that minimize the expected total cost subject to the reliability requirement are developed.


90B25 Reliability, availability, maintenance, inspection in operations research
Full Text: DOI


[1] Akaike, H., A new look at statistical model identification, IEEE Transactions on Automatic Control, 19, 716-723 (1974) · Zbl 0314.62039
[2] Ehrlich, W.; Prasanna, B.; Stampfel, J.; Wu, J., Determining the cost of a stop-testing decision, IEEE Software, March, 33-42 (1993)
[3] Goel, A. L.; Okumoto, K., Time-dependent error-detection rate model for software and other performance measures, IEEE Transaction on Reliability, 28, 206-211 (1979) · Zbl 0409.68009
[4] Hossain, S. A.; Ram, C. D., Estimating the parameters of a non-homogeneous Poisson process model for software reliability, IEEE Transactions on Reliability, 42, 4, 604-612 (1993)
[5] Kareer, N.; Kapur, P. K.; Grover, P. S., An S-shaped software reliability growth model with two types of errors, Microelectronics and Reliability - An International Journal, 30, 1085-1090 (1990)
[6] Kapur, P. K.; Bhalla, V. K., Optimal release policies for a flexible software reliability growth model, Reliability Engineering and System Safety Journal, 35, 45-54 (1992) · Zbl 0742.60083
[7] Leung, Y. W., Optimal software release time with a given cost budget, Journal of Systems and Software, 17, 233-242 (1992)
[8] Misra, P. N., Software reliability analysis, IBM Systems Journal, 22, 262-270 (1983)
[9] Ohba, M., Software reliability analysis models, IBM Journal of Research Development, 28, 428-443 (1984)
[10] Ohba, M., Inflection S-shaped software reliability growth models, (Osaki, S.; Hatoyama, Y., Stochastic Models in Reliability Theory (1984), Springer: Springer Berlin), 144-162
[11] Ohba, M.; Chou, X. M., Does imperfect debugging affect software reliability growth?, (Proceedings of the 11th IEEE International Conference on Software Engineering (1989)), 237-244
[12] Ohba, M.; Yamada, S., S-shaped software reliability growth models, (Proceedings of the 4th International Conference on Reliability and Maintainability (1984)), 430-436
[13] Ohtera, H.; Yamada, S., Optimal allocation and control problems for software-testing resources, IEEE Transactions on Reliability, 39, 171-176 (1990) · Zbl 0709.68011
[14] Pham, H., Software Reliability and Testing (1995), IEEE Computer Society Press: IEEE Computer Society Press Silver Spring, MD · Zbl 0822.68019
[15] Pham, H.; Nordmann, L.; Zhang, X., A General imperfect software debugging model with S-shaped fault detection rate, IEEE Transactions on Reliability, 48, 2, 169-175 (1999)
[16] H. Pham, Software reliability assessment: Imperfect debugging and multiple fault types in software development, EG&G-RAAM-10737, Idaho National Laboratory, 1993; H. Pham, Software reliability assessment: Imperfect debugging and multiple fault types in software development, EG&G-RAAM-10737, Idaho National Laboratory, 1993
[17] Pham, H.; Zhang, X., An NHPP software reliability model and its comparison, International Journal of Reliability, Quality and Safety Engineering, 4, 3, 269-282 (1997)
[18] Pham, H.; Zhang, X., A software cost model with warranty and risk costs, IEEE Transactions on Computers, 48, 1, 71-75 (1999)
[19] Pham, H., Software Reliability (2000), Springer: Springer Berlin · Zbl 0943.68007
[20] Pham, H.; Zhang, X., Software release policies with gain in reliability justifying the costs, Annals of Software Engineering, 8, 147-166 (1999)
[21] Pham, H., Software reliability, (Webster, J. G., Wiley Encyclopedia of Electrical and Electronics Engineering (2000), Wiley: Wiley New York) · Zbl 0822.68019
[22] Pham, H.; Wang, H., Software reliability and cost modeling by a quasi renewal process, IEEE Transactions on Systems Man, and Cybernetics, 31, 6, 623-631 (2001)
[23] Wood, A., Predicting software reliability, IEEE Computer, 11, 69-77 (1996)
[24] Yamada, S.; Onha, M.; Osaki, S., S-shaped reliability growth modeling for software error detection, IEEE Transactions on Reliability, 12, 475-484 (1983)
[25] Yamada, S.; Ohtera, H.; Ohba, M., Testing-domain dependent software reliability model, Computers and Mathematics with Applications, 24, 79-86 (1992) · Zbl 0781.62152
[26] Yamada, S.; Ohtera, H.; Narihisa, H., Software reliability growth models with testing effort, IEEE Transactions on Reliability, 4, 19-23 (1986)
[27] Yamada, S., Software quality/reliability measurement and assessment: Software reliability growth models and data analysis, Journal of Information Processing, 14, 3, 254-266 (1991)
[28] Yamada, S.; Tokuno, K.; Osaki, S., Imperfect debugging models with fault introduction rate for software reliability assessment, International Journal of Systems Science, 23, 12 (1992) · Zbl 0795.68045
[29] Yamada, S.; Osaki, S., Optimal software release policies with simultaneous cost and reliability criteria, European Journal of Operational Research, 31, 1, 46-51 (1987) · Zbl 0614.90041
[30] Zhang, X.; Pham, H., A software cost model with error removal times and risk costs, International Journal of Systems Science, 29, 40, 435-442 (1998)
[31] Handbook of Mathematical for Mathematicians, Scientists, Engineers (1980), Research and Education Association: Research and Education Association New York
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.