×

Global testing against sparse alternatives in time-frequency analysis. (English) Zbl 1359.62032

The authors’ aims of this paper are to establish a number of results on global testing for periodicity. They construct a new test based on an over-sampled periodogram that adapts to a growing number of sinusoids with general frequences. The focus of this work is to find the asymptotic optimality of this method. Their main result guarantees that as long as frequencies of the complex sinusoids in the mean of \(y=X\beta+z\) obey some minimum separation condition, this over-sampling rate leads to an asymptotically optimal global test where \(X\in \mathbb C^{N\times p}\) with \(p\gg N\) is an extended Fourier transform matrix and \(z\) represents white noise.

MSC:

62C20 Minimax procedures in statistical decision theory
62F03 Parametric hypothesis testing
62F05 Asymptotic properties of parametric tests
62G30 Order statistics; empirical distribution functions
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)

Software:

BLOOMP
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Ahdesmaki, M., Lahdesmaki, H., Pearson, R., Huttunen, H. and Yli-Harja, O. (2005). Robust detection of periodic time series measured from biological systems. BMC Bioinformatics 6 117.
[2] Ahdesmaki, M., Lahdesmaki, H. and Yli-Harja, O. (2007). Robust Fisher’s test for periodicity detection in noisy biological time series. In GENSIPS , IEEE International Workshop on 1-4. Tuusula, Finland.
[3] Alexander, K. S. (1987). Rates of growth and sample moduli for weighted empirical processes indexed by sets. Probab. Theory Related Fields 75 379-423. · Zbl 0596.60029 · doi:10.1007/BF00318708
[4] Arias-Castro, E., Candès, E. J., Helgason, H. and Zeitouni, O. (2008). Searching for a trail of evidence in a maze. Ann. Statist. 36 1726-1757. · Zbl 1143.62006 · doi:10.1214/07-AOS526
[5] Arias-Castro, E., Candès, E. J. and Plan, Y. (2011). Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist. 39 2533-2556. · Zbl 1231.62136 · doi:10.1214/11-AOS910
[6] Arias-Castro, E., Donoho, D. L. and Huo, X. (2005). Near-optimal detection of geometric objects by fast multiscale methods. IEEE Trans. Inform. Theory 51 2402-2425. · Zbl 1282.94014 · doi:10.1109/TIT.2005.850056
[7] Bhaskar, B. N., Tang, G. and Recht, B. (2013). Atomic norm denoising with applications to line spectral estimation. IEEE Trans. Signal Process. 61 5987-5999. · Zbl 1394.94079 · doi:10.1109/TSP.2013.2273443
[8] Bølviken, E. (1983). New tests of significance in periodogram analysis. Scand. J. Stat. 10 1-9.
[9] Brockwell, P. J. and Davis, R. A. (2009). Times Series : Theory and Methods , 2nd ed. Springer, Berlin. · Zbl 1169.62074
[10] Cadzow, J. (1988). Signal enhancement-a composite property mapping algorithm. IEEE Trans. Acoust. Speech Signal Process. 36 49-62. · Zbl 0649.93059 · doi:10.1109/29.1488
[11] Cai, T., Eldar, Y. C. and Li, X. (2016). Supplement to “Global testing against sparse alternatives in time-frequency analysis.” . · Zbl 1359.62032 · doi:10.1214/15-AOS1412
[12] Cai, T. T., Jeng, X. J. and Jin, J. (2011). Optimal detection of heterogeneous and heteroscedastic mixtures. J. R. Stat. Soc. Ser. B Stat. Methodol. 73 629-662. · Zbl 1228.62020 · doi:10.1111/j.1467-9868.2011.00778.x
[13] Cai, T. T., Jin, J. and Low, M. G. (2007). Estimation and confidence sets for sparse normal mixtures. Ann. Statist. 35 2421-2449. · Zbl 1360.62113 · doi:10.1214/009053607000000334
[14] Cai, T. T. and Wu, Y. (2014). Optimal detection of sparse mixtures against a given null distribution. IEEE Trans. Inform. Theory 60 2217-2232. · Zbl 1360.94108 · doi:10.1109/TIT.2014.2304295
[15] Candès, E. J. and Fernandez-Granda, C. (2013). Super-resolution from noisy data. J. Fourier Anal. Appl. 19 1229-1254. · Zbl 1312.94015 · doi:10.1007/s00041-013-9292-3
[16] Candès, E. J. and Fernandez-Granda, C. (2014). Towards a mathematical theory of super-resolution. Comm. Pure Appl. Math. 67 906-956. · Zbl 1350.94011 · doi:10.1002/cpa.21455
[17] Chen, J. (2005). Identification of significant genes in microarray gene expression data. BMC Bioinformatics 6 286.
[18] Chen, S. S. and Donoho, D. L. (1998). Application of basis pursuit in spectrum estimation. In Conference on Acoustics , Speech and Signal Processing , Vol. 3. Seattle, WA.
[19] Chiu, S.-T. (1989). Detecting periodic components in a white Gaussian time series. J. Roy. Statist. Soc. Ser. B 51 249-259. · Zbl 0669.62088
[20] Davies, R. B. (1987). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 74 33-43. · Zbl 0612.62023
[21] Dicker, L. H. (2014). Variance estimation in high-dimensional linear models. Biometrika 101 269-284. · Zbl 1452.62495 · doi:10.1093/biomet/ast065
[22] Djurić, P. M. (1996). A model selection rule for sinusoids in white Gaussian noise. IEEE Trans. Signal Process. 44 1744-1751.
[23] Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962-994. · Zbl 1092.62051 · doi:10.1214/009053604000000265
[24] Donoho, D. L. (1992). Superresolution via sparsity constraints. SIAM J. Math. Anal. 23 1309-1331. · Zbl 0769.42007 · doi:10.1137/0523074
[25] Duarte, M. F. and Baraniuk, R. G. (2013). Spectral compressive sensing. Appl. Comput. Harmon. Anal. 35 111-129. · Zbl 1336.94016 · doi:10.1016/j.acha.2012.08.003
[26] Fannjiang, A. and Liao, W. (2012). Coherence pattern-guided compressive sensing with unresolved grids. SIAM J. Imaging Sci. 5 179-202. · Zbl 1250.65035 · doi:10.1137/110838509
[27] Fisher, R. A. (1929). Tests of significance in harmonic analysis. Proceedings of Royal Society , Ser. A 125 54-59.
[28] Fuller, W. A. (1976). Introduction to Statistical Time Series . Wiley, New York. · Zbl 0353.62050
[29] Giné, E. and Koltchinskii, V. (2006). Concentration inequalities and asymptotic results for ratio type empirical processes. Ann. Probab. 34 1143-1216. · Zbl 1152.60021 · doi:10.1214/009117906000000070
[30] Glynn, E. F., Chen, J. and Mushegian, A. R. (2006). Detecting periodic patterns in unevenly spaced gene expression time series using Lomb-Scargle periodograms. Bioinformatics 22 310-316.
[31] Hall, P. and Jin, J. (2010). Innovated higher criticism for detecting sparse signals in correlated noise. Ann. Statist. 38 1686-1732. · Zbl 1189.62080 · doi:10.1214/09-AOS764
[32] Hu, L., Shi, Z., Zhou, J. and Fu, Q. (2012). Compressed sensing of complex sinusoids: An approach based on dictionary refinement. IEEE Trans. Signal Process. 60 3809-3822. · Zbl 1393.94686 · doi:10.1109/TSP.2012.2193392
[33] Hua, Y. and Sarkar, T. K. (1990). Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise. IEEE Trans. Acoust. Speech Signal Process. 38 814-824. · Zbl 0706.62094 · doi:10.1109/29.56027
[34] Ingster, Yu. I. (1998). Minimax detection of a signal for \(l^{n}\)-balls. Math. Methods Statist. 7 401-428 (1999). · Zbl 1103.62312
[35] Ingster, Y. I., Tsybakov, A. B. and Verzelen, N. (2010). Detection boundary in sparse regression. Electron. J. Stat. 4 1476-1526. · Zbl 1329.62314 · doi:10.1214/10-EJS589
[36] Juditsky, A. and Nemirovski, A. (2015). On detecting harmonic oscillations. Bernoulli 21 1134-1165. · Zbl 1385.60051 · doi:10.3150/14-BEJ600
[37] Kay, S. M. (1998). Fundamentals of Statistical Signal Processing : Detection Theory . Prentice Hall, Upper Saddle River, NJ.
[38] Knight, W. C., Pridham, R. G. and Kay, S. M. (1981). Digital signal processing for sonar. Proc. IEEE 69 1451-1506.
[39] Leadbetter, M. R., Lindgren, G. and Rootzén, H. (1983). Extremes and Related Properties of Random Sequences and Processes . Springer, New York. · Zbl 0518.60021
[40] Li, J. and Siegmund, D. (2015). Higher criticism: \(p\)-values and criticism. Ann. Statist. 43 1323-1350. · Zbl 1320.62039 · doi:10.1214/15-AOS1312
[41] Li, W. V. and Shao, Q.-M. (2002). A normal comparison inequality and its applications. Probab. Theory Related Fields 122 494-508. · Zbl 1004.60031 · doi:10.1007/s004400100176
[42] Nadler, B. and Kontorovich, A. (2011). Model selection for sinusoids in noise: Statistical analysis and a new penalty term. IEEE Trans. Signal Process. 59 1333-1345. · Zbl 1392.94643 · doi:10.1109/TSP.2011.2105482
[43] Prony, R. (1795). Essai expérimental et analytique: Sur les lois de la dilatabilité de fluides élastique et sur celles de la force expansive de la vapeur de l’alkool, à différentes températures. J. Éc. Polytech. 1 24-76.
[44] Quinn, B. G. and Hannan, E. J. (2001). The Estimation and Tracking of Frequency. Cambridge Series in Statistical and Probabilistic Mathematics 9 . Cambridge Univ. Press, Cambridge. · Zbl 0969.62060 · doi:10.1017/CBO9780511609602
[45] Quinn, B. G. and Kootsookos, P. J. (1994). Threshold behavior of the maximum likelihood estimator of frequency. IEEE Trans. Signal Process. 42 3291-3294.
[46] Roy, R. and Kailath, T. (1990). ESPRIT-estimation of signal parameters via rotational invariance techniques. In Signal Processing , Part II. IMA Vol. Math. Appl. 23 369-411. Springer, New York. · Zbl 0701.93090 · doi:10.1007/978-1-4684-7095-6_20
[47] Rubin-Delanchy, P. and Walden, A. T. (2008). Kinematics of complex-valued time series. IEEE Trans. Signal Process. 56 4189-4198. · Zbl 1390.94386 · doi:10.1109/TSP.2008.926106
[48] Schmidt, R. (1986). Multiple emitter location and signal parameter estimation. IEEE Trans. Antennas and Propagation 34 276-280.
[49] Siegel, A. F. (1980). Testing for periodicity in a time series. J. Amer. Statist. Assoc. 75 345-348. · Zbl 0462.62069 · doi:10.2307/2287456
[50] Skolnik, M. I. (1980). Introduction to Radar Systems . McGraw-Hill, New York.
[51] Sykulski, A. M., Olhede, S. C., Lilly, J. M. and Early, J. J. (2013). The Whittle likelihood for complex-valued time series.
[52] Tang, G., Bhaskar, B. N. and Recht, B. (2015). Near minimax line spectral estimation. IEEE Trans. Inform. Theory 61 499-512. · Zbl 1359.94181 · doi:10.1109/TIT.2014.2368122
[53] van Zuijlen, M. C. A. (1978). Properties of the empirical distribution function for independent nonidentically distributed random variables. Ann. Probab. 6 250-266. · Zbl 0396.60040 · doi:10.1214/aop/1176995571
[54] Van Trees, H. L. (1992). Detection , Estimation , and Modulation Theory , Part 3: Radar-Sonar Signal Processing and Gaussian Signals in Noise . Wiley, New York.
[55] Whalen, A. D. (1995). Detection of Signals in Noise . Academic Press, New York.
[56] Wichert, S., Fokianos, K. and Strimmer, K. (2004). Identifying periodically expressed transcripts in microarray time series data. Bioinformatics 20 5-20.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.