Most recent changepoint detection in censored panel data. (English) Zbl 07315566

Siddiqa, Hajra; Ali, Sajid; Shah, Ismail

Comput. Stat. 36, No. 1, 515-540 (2021).

Summary: This study aims to detect the most recent changepoint in censored panel data by ignoring dependence within and between segments as well as taking into account the serial autocorrelation. A comparison of different methods to detect the most recent changepoint for censored data is presented. Different censoring rates such as 20%, 50%, and 90% in the case of right and left censoring while (10%, 10%), (25%, 25%) and (40%, 50%) for interval censoring are considered. Further, we use most recent changepoint (MRC), double cumulative sum binary segmentation, non parametric changepoint detection (ECP), multiple changepoints in multivariate time series, analyzing each series in the panel independently, and analyzing aggregated data (AGG) methods. It is observed that different censoring rates have a significant effect on the detection of changepoints in high dimensional data. It is also noticed that the MRC method outperforms the competing methods considered in this study. In addition to investigating the impact of penalties, the performance of MRC and AGG methods is also compared using water quality data of the Niagara River. Also, a data set related to survival time of stroke patients is also a part of this study. An R package “cpcens” is available in comprehensive R archive network to replicate the results of this article.

MSC:

62-08	Computational methods for problems pertaining to statistics
62M10	Time series, auto-correlation, regression, etc. in statistics (GARCH)

Keywords:

change point; panel censored data; high dimensional data; CUSUM; binary segmentation; cost function

Software:

TSA; R; cpcens; wbs; ecp

References:

[1]	Aston JA, Kirch C (2014) Efficiency of change point tests in high dimensional settings. arXiv preprint arXiv: 1409.1771
[2]	Bardwell L (2018) Efficient search methods for high dimensional time-series. Ph.D. thesis, Lancaster University
[3]	Bardwell, L.; Fearnhead, P.; Eckley, IA; Smith, S.; Spott, M., Most recent changepoint detection in panel data, Technometrics, 61, 1, 88-98 (2019)
[4]	Bellman, RE; Dreyfus, SE, Applied dynamic programming (2015), Princeton: Princeton University Press, Princeton · Zbl 0106.34901
[5]	Cao, H.; Biao Wu, W., Changepoint estimation: another look at multiple testing problems, Biometrika, 102, 4, 974-980 (2015) · Zbl 1390.62138
[6]	Charikar, M.; Guha, S.; Tardos, É.; Shmoys, DB, A constant-factor approximation algorithm for the k-median problem, J Comput Syst Sci, 65, 1, 129-149 (2002) · Zbl 1023.90037
[7]	Cho, H., Change-point detection in panel data via double CUSUM statistic, Electron J Stat, 10, 2, 2000-2038 (2016) · Zbl 1397.62301
[8]	Cho, H.; Fryzlewicz, P., Multiple-change-point detection for high dimensional time series via sparsified binary segmentation, J R Stat Soc Ser B (Stat Methodol), 77, 2, 475-507 (2015) · Zbl 1414.62356
[9]	Cohen, AC, Truncated and censored samples: theory and applications (2016), Boca Raton: CRC Press, Boca Raton
[10]	Coolen, F.; Yan, K., Nonparametric predictive inference with right-censored data, J Stat Plan Inference, 126, 1, 25-54 (2004) · Zbl 1092.62058
[11]	Cryer, J.; Chan, K., Time series analysis: with applications in R (2008), Berlin: Springer, Berlin
[12]	Davis, RA; Lee, TCM; Rodriguez-Yam, GA, Structural break estimation for nonstationary time series models, J Am Stat Assoc, 101, 473, 223-239 (2006) · Zbl 1118.62359
[13]	Fearnhead, P.; Rigaill, G., Changepoint detection in the presence of outliers, J Am Stat Assoc, 114, 169-183 (2019) · Zbl 1478.62238
[14]	Fryzlewicz, P., Wild binary segmentation for multiple change-point detection, Ann Stat, 42, 6, 2243-2281 (2014) · Zbl 1302.62075
[15]	Grünwald, PD, The minimum description length principle (2007), Cambridge: MIT Press, Cambridge
[16]	Haynes, K.; Eckley, IA; Fearnhead, P., Computationally efficient changepoint detection for a range of penalties, J Comput Graph Stat, 26, 1, 134-143 (2017)
[17]	Helsel, DR, Statistics for censored environmental data using Minitab and R (2011), New York: Wiley, New York
[18]	Hewett, P.; Ganser, GH, A comparison of several methods for analyzing censored data, Ann Occup Hyg, 51, 7, 611-632 (2007)
[19]	Horváth, L.; Hušková, M., Change-point detection in panel data, J Time Ser Anal, 33, 4, 631-648 (2012) · Zbl 1282.62181
[20]	James NA, Matteson DS (2013) ecp: An R package for nonparametric multiple change point analysis of multivariate data. arXiv preprint arXiv:1309.3295
[21]	Jandhyala, V.; Fotopoulos, S.; MacNeill, I.; Liu, P., Inference for single and multiple change-points in time series, J Time Ser Anal, 34, 4, 423-446 (2013) · Zbl 1275.62061
[22]	Killick, R.; Fearnhead, P.; Eckley, IA, Optimal detection of changepoints with a linear computational cost, J Am Stat Assoc, 107, 500, 1590-1598 (2012) · Zbl 1258.62091
[23]	Kirch, C.; Muhsal, B.; Ombao, H., Detection of changes in multivariate time series with application to EEG data, J Am Stat Assoc, 110, 511, 1197-1216 (2015) · Zbl 1378.62072
[24]	Lavielle, M., Using penalized contrasts for the change-point problem, Sig Process, 85, 8, 1501-1510 (2005) · Zbl 1160.94341
[25]	Lavielle, M.; Moulines, E., Least-squares estimation of an unknown number of shifts in a time series, J Time Ser Anal, 21, 1, 33-59 (2000) · Zbl 0974.62070
[26]	Lavielle, M.; Teyssiere, G., Detection of multiple change-points in multivariate time series, Lith Math J, 46, 3, 287-306 (2006) · Zbl 1138.62051
[27]	Leung, K-M; Elashoff, RM; Afifi, AA, Censoring issues in survival analysis, Annu Rev Public Health, 18, 1, 83-104 (1997)
[28]	Ma, TF; Yau, CY, A pairwise likelihood-based approach for changepoint detection in multivariate time series models, Biometrika, 103, 2, 409-421 (2016) · Zbl 1499.62314
[29]	Maidstone, R.; Hocking, T.; Rigaill, G.; Fearnhead, P., On optimal multiple changepoint algorithms for large data, Stat Comput, 27, 2, 519-533 (2017) · Zbl 06697671
[30]	Matteson, DS; James, NA, A nonparametric approach for multiple change point analysis of multivariate data, J Am Stat Assoc, 109, 505, 334-345 (2014) · Zbl 1367.62260
[31]	Mei Y (2011) Quickest detection in censoring sensor networks. In: 2011 IEEE international symposium on information theory proceedings (ISIT), pp 2148-2152. IEEE
[32]	Miller, RG Jr, Survival analysis (2011), New York: Wiley, New York
[33]	Mohammad NM (2014) Censored Time Series Analysis. Electronic Thesis and Dissertation Repository. 2489. https://ir.lib.uwo.ca/etd/2489
[34]	Nemhauser, G.; Wolsey, L., Integer programming and combinatorial optimization (1988), New York: Wiley, New York · Zbl 0652.90067
[35]	Park, JW; Genton, MG; Ghosh, SK, Censored time series analysis with autoregressive moving average models, Can J Stat, 35, 1, 151-168 (2007) · Zbl 1124.62061
[36]	Preuss, P.; Puchstein, R.; Dette, H., Detection of multiple structural breaks in multivariate time series, J Am Stat Assoc, 110, 510, 654-668 (2015) · Zbl 1373.62454
[37]	Reese, J., Solution methods for the p-median problem: an annotated bibliography, Netw Int J, 48, 3, 125-142 (2006) · Zbl 1133.90357
[38]	Robinson PM (1980) Estimation and forecasting for time series containing censored or missing observations. In: Anderson OD (ed) Time series. North Holland, Amsterdam, New York, pp 167-182. Proceedings of the international conference held at Nottingham University
[39]	Taha, HA, Operations research: an introduction (2017), London: Pearson, London · Zbl 0227.90002
[40]	Teitz, MB; Bart, P., Heuristic methods for estimating the generalized vertex median of a weighted graph, Oper Res, 16, 5, 955-961 (1968) · Zbl 0165.22804
[41]	Vert J-P, Bleakley K (2010) Fast detection of multiple change-points shared by many signals using group LARS. In: Advances in neural information processing systems, pp 2343-2351
[42]	Wang, T.; Samworth, RJ, High dimensional change point estimation via sparse projection, J R Stat Soc Ser B (Stat Methodol), 80, 1, 57-83 (2018) · Zbl 1439.62199
[43]	Wooldridge, JM, Econometric analysis of cross section and panel data (2010), Cambridge: MIT Press, Cambridge · Zbl 1327.62009
[44]	Xie, Y.; Siegmund, D., Sequential multi-sensor change-point detection, Ann Stat, 41, 670-692 (2013) · Zbl 1267.62084
[45]	Yao, Y-C, Approximating the distribution of the maximum likelihood estimate of the change-point in a sequence of independent random variables, Ann Stat, 15, 3, 1321-1328 (1987) · Zbl 0651.62017
[46]	Zeger, SL; Brookmeyer, R., Regression analysis with censored autocorrelated data, J Am Stat Assoc, 81, 395, 722-729 (1986) · Zbl 0623.62084
[47]	Zhang, NR; Siegmund, DO, A modified bayes information criterion with applications to the analysis of comparative genomic hybridization data, Biometrics, 63, 1, 22-32 (2007) · Zbl 1206.62174