Filtering and tracking survival propensity (reconsidering the foundations of reliability). (English) Zbl 1442.62221

Summary: The work described here was motivated by the need to address a long standing problem in engineering, namely, the tracking of reliability growth. An archetypal scenario is the performance of software as it evolves over time. Computer scientists are challenged by the task of when to release software. The same is also true for complex engineered systems like aircraft, automobiles and ballistic missiles. Tracking problems also arise in actuarial science, biostatistics, cancer research and mathematical finance.
A natural approach for addressing such problems is via the control theory methods of filtering, smoothing and prediction. But to invoke such methods, one needs a proper philosophical foundation, and this has been lacking. The first three sections of this paper endeavour to fill this gap. A consequence is the point of view proposed here, namely, that reliability not be interpreted as a probability. Rather, reliability should be conceptualized as a dynamically evolving propensity in the sense of Pierce and Popper. Whereas propensity is to be taken as an undefined primitive, it manifests as a chance (or frequency) in the sense of de Finetti. The idea of looking at reliability as a propensity also appears in the philosophical writings of Kolmogorov. Furthermore, survivability which quantifies ones uncertainty about a propensity should be the metric of performance that needs to be tracked. The first part of this paper is thus a proposal for a paradigm shift in the manner in which one conceptualizes reliability, and by extension, survival analysis. This message is also germane to other areas of applied probability and statistics, like queueing, inventory and time series analysis.
The second part of this paper is technical. Its purpose is to show how the philosophical material of the first part can be incorporated into a framework that leads to a methodological package. To do so, we focus on the problem which motivated the first part, and develop a mathematical model for describing the evolution of an item’s propensity to survive. The item could be a component, a system or a biological entity. The proposed model is based on the notion of competing risks. Models like this also appear in biostatistics under the label of cure models. Whereas the competing risks scenario is instructive, it is not the only way to describe the phenomenon of growth; its use here is illustrative. All the same, one of its virtues is that it paves the path towards a contribution to the state of the art of filtering by considering the case of censored observations. Even though censoring is the hallmark of survival analysis, it could also arise in time series analysis and control theory, making the development here of a broader and more general appeal.


62N05 Reliability and life testing
62A01 Foundations and philosophical topics in statistics
62M20 Inference from stochastic processes and prediction
62P30 Applications of statistics in engineering and industry; control charts
Full Text: DOI Euclid


[1] Aalen, O. O. (1988). Heterogenity in survival analysis. Stat. Med.7 1121-1137.
[2] Barlow, R. E. and Mendel, M. B. (1992). De Finetti-type representations for life distributions. J. Amer. Statist. Assoc.87 1116-1122. · Zbl 0764.62077
[3] Barlow, R. E. and Proschan, F. (1965). Mathematical Theory of Reliability. Wiley, New York-London-Sydney. With Contributions by Larry C. Hunter. · Zbl 0132.39302
[4] Barlow, R. E. and Proschan, F. (1975). Statistical Theory of Reliability and Life Testing: Probability Models. Holt, Rinehart and Winston, Inc., New York. · Zbl 0379.62080
[5] Bather, J. A. (1965). Invariant conditional distributions. Ann. Math. Stat.36 829-846. · Zbl 0143.19602
[6] Block, H. W. and Savits, T. H. (1997). Burn-in. Statist. Sci.12 1-13.
[7] Chen, M.-H., Ibrahim, J. G. and Sinha, D. (1999). A new Bayesian model for survival data with a surviving fraction. J. Amer. Statist. Assoc.94 909-919. · Zbl 0996.62019
[8] Clayton, D. G. (1991). A Monte Carlo method for Bayesian inference in frailty models. Biometrics47 467-485.
[9] Clayton, D. and Cuzick, J. (1985). Multivariate generalizations of the proportional hazards model. J. Roy. Statist. Soc. Ser. A148 82-117. · Zbl 0581.62086
[10] Cox, D. R. (1972). Regression models and life-tables. J. R. Stat. Soc. Ser. B. Stat. Methodol.34 187-220. With discussion by F. Downton, Richard Peto, D. J. Bartholomew, D. V. Lindley, P. W. Glassborow, D. E. Barton, Susannah Howard, B. Benjamin, John J. Gart, L. D. Meshalkin, A. R. Kagan, M. Zelen, R. E. Barlow, Jack Kalbfleisch, R. L. Prentice and Norman Breslow, and a reply by D. R. Cox. · Zbl 0243.62041
[11] de Finetti, B. (1937). La prévision : Ses lois logiques, ses sources subjectives. Ann. Inst. Henri Poincaré7 1-68. · Zbl 0017.07602
[12] de Groot, M. H. (1988). A Bayesian view of assessing uncertainty and comparing expert opinion. J. Statist. Plann. Inference20 295-306. · Zbl 0666.62003
[13] Diaconis, P. and Freedman, D. (1987). A dozen de Finetti-style results in search of a theory. Ann. Inst. Henri Poincaré Probab. Stat.23 397-423. · Zbl 0619.60039
[14] Dubey, S. D. (1966). Compound Pascal distributions. Ann. Inst. Statist. Math.18 357-365. · Zbl 0151.24005
[15] Escobar, L. A. and Meeker, W. Q. (2006). A review of accelerated test models. Statist. Sci.21 552-577. · Zbl 1129.62090
[16] Fetzer, J. H. (1981). Scientific Knowledge: Causation, Explanation, and Corroboration. Boston Stud. Philos. Sci.69. D. Reidel, Dordrecht.
[17] Gillies, D. (2000). Varieties of propensity. British J. Philos. Sci.51 807-835.
[18] Good, I. J. (1950). Probability and the Weighing of Evidence. Charles Griffin & Co., Ltd., London. · Zbl 0036.08402
[19] Good, I. J. and Card, W. I. (1971). The diagnostic process with special reference to errors. Methods Inf. Med.10 176-188.
[20] Hamada, M. S., Wilson, A. G., Reese, C. S. and Martz, H. F. (2008). Bayesian Reliability. Springer, New York. · Zbl 1165.62074
[21] Hewitt, E. and Savage, L. J. (1955). Symmetric measures on Cartesian products. Trans. Amer. Math. Soc.80 470-501. · Zbl 0066.29604
[22] Howson, C. and Urbach, P. (2006). Scientific Reasoning: The Bayesian Approach. Open Court, Chicago, IL.
[23] Humphreys, P. (1985). Why propensities cannot be probabilities. Philos. Rev.94 557-570.
[24] Kendall, M. G. (1949). On the reconciliation of theories of probability. Biometrika36 101-116. · Zbl 0032.28901
[25] Kolmogorov, A. N. (1969). The theory of probability. In Mathematics: Its Content, Methods and Meaning, Vol. 2, Part 3. (A. D. Aleksandrov, A. N. Kolmogorov and M. A. Lavrentav, eds.). MIT Press, Cambridge, MA. · Zbl 1221.01158
[26] Lindley, D. V. (1990). The 1988 Wald memorial lectures: The present position in Bayesian statistics. Statist. Sci.5 44-89. With comments and a rejoinder by the author. · Zbl 0955.62515
[27] Lindley, D. V. and Singpurwalla, N. D. (1986). Multivariate distributions for the life lengths of components of a system sharing a common environment. J. Appl. Probab.23 418-431. · Zbl 0603.60084
[28] Lindley, D. V. and Singpurwalla, N. D. (2002). On exchangeable, causal and cascading failures. Statist. Sci.17 209-219. · Zbl 1014.62114
[29] Martin, O. S. (2012). A Dynamic Competing Risk Model for Filtering Reliability and Tracking Survivability. ProQuest LLC, Ann Arbor, MI. Ph.D. thesis, The George Washington University.
[30] Meeker, W. Q. and Escobar, L. A. (1998). Statistical Methods for Reliability Data. Wiley, New York. · Zbl 0949.62086
[31] Meinhold, R. and Singpurwalla, N. D. (1983). Understanding the Kalman filter. Amer. Statist.37 123-127.
[32] Miller, R. W. (1975). Propensity: Popper or peirce? British J. Philos. Sci. 123-132. · Zbl 0353.00010
[33] Popper, K. R. (1957). The propensity interpretation of the calculus of probability, and the quantum theory. In Observation and Interpretation in the Philosophy of Physics (S. Korner, ed.). Dover, Mineola, NY. · Zbl 0078.43802
[34] Popper, K. R. (1959). The propensity interpretation of probability. British J. Philos. Sci.10 25-42.
[35] Singpurwalla, N. D. (1988). Foundational issues in reliability and risk analysis. SIAM Rev.30 264-282. · Zbl 0686.62078
[36] Singpurwalla, N. D. (1995). Survival in dynamic environments. Statist. Sci.10 86-113. · Zbl 1148.62314
[37] Singpurwalla, N. D. (2002). Some cracks in the empire of chance (flaws in the foundations of reliability). Int. Stat. Rev.70 53-65. · Zbl 1330.62051
[38] Singpurwalla, N. D. (2006). Reliability and Risk: A Bayesian Perspective. Wiley, Chichester. · Zbl 1152.62070
[39] Singpurwalla, N. D. (2007). Betting on residual life: The caveats of conditioning. Statist. Probab. Lett.77 1354-1361. · Zbl 1115.62010
[40] Singpurwalla, N. D. (2010). Comment: “Reliability growth management metrics and statistical methods for discrete-use systems” [MR2789242]. Technometrics52 396-397.
[41] Singpurwalla, N. D. and Wilson, P. (2004). When can finite testing ensure infinite trustworthiness? J. Iran. Stat. Soc. (JIRSS) 3 1-37. · Zbl 06657078
[42] Smith, A. F. M. (1981). On random sequences with centred spherical symmetry. J. R. Stat. Soc. Ser. B. Stat. Methodol.43 208-209. · Zbl 0471.62016
[43] Spizzichino, F. (2001). Subjective Probability Models for Lifetimes. Monographs on Statistics and Applied Probability91. Chapman & Hall/CRC, Boca Raton, FL. · Zbl 1078.62530
[44] Tsodikov, A. D., Ibrahim, J. G. and Yakovlev, A. Y. (2003). Estimating cure rates from survival data: An alternative to two-component mixture models. J. Amer. Statist. Assoc.98 1063-1078.
[45] von Mises, R. (1941). On the foundations of probability and statistics. Ann. Math. Stat.12 191-205. · JFM 67.0452.03
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.