Global testing against sparse alternatives under Ising models. (English) Zbl 1407.62160

This paper deals with Ising models of the form \[ \mathbb P_{\mathbf Q, \pmb \mu} \left( \mathbf X = \mathbf x\right) = \frac{1}{Z \left(\mathbf Q, \pmb \mu\right)} \exp\left(\frac12 \mathbf x^\intercal \mathbf Q \mathbf x + \pmb \mu^\intercal \mathbf x\right), \qquad \mathbf x \in \left\{\pm 1\right\}^n \] and the question how to test weather \( \pmb \mu = \mathbf 0\) or not from random observations \(\mathbf X = \left(X_1,...,X_n\right)^{\intercal} \in \left\{\pm 1\right\}^n\). The considered alternatives are assumed to be sparse in a suitable sense, and the authors study the impact of the dependency describing matrix \(\mathbf Q\) onto the detection threshold of the problem.
For different sub-classes of Ising models, the authors construct explicit testing procedures and prove that they are asymptotically rate optimal. Also a comparison between different models is provided.


62G20 Asymptotic properties of nonparametric inference
62G10 Nonparametric hypothesis testing
62P35 Applications of statistics to physics
60K35 Interacting random processes; statistical mechanics type models; percolation theory
82B44 Disordered systems (random Ising models, random Schrödinger operators, etc.) in equilibrium statistical mechanics
62C20 Minimax procedures in statistical decision theory
Full Text: DOI arXiv Euclid


[1] Addario-Berry, L., Broutin, N., Devroye, L. and Lugosi, G. (2010). On combinatorial testing problems. Ann. Statist.38 3063–3092. · Zbl 1200.62059
[2] Arias-Castro, E., Candès, E. J. and Plan, Y. (2011). Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist.39 2533–2556. · Zbl 1231.62136
[3] Arias-Castro, E., Donoho, D. L. and Huo, X. (2005). Near-optimal detection of geometric objects by fast multiscale methods. IEEE Trans. Inform. Theory51 2402–2425. · Zbl 1282.94014
[4] Arias-Castro, E. and Wang, M. (2015). The sparse Poisson means model. Electron. J. Stat.9 2170–2201. · Zbl 1337.62088
[5] Arias-Castro, E., Candès, E. J., Helgason, H. and Zeitouni, O. (2008). Searching for a trail of evidence in a maze. Ann. Statist.36 1726–1757. · Zbl 1143.62006
[6] Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. Roy. Statist. Soc. Ser. B36 192–236. · Zbl 0327.60067
[7] Besag, J. (1975). Statistical analysis of non-lattice data. Amer. Statist. 179–195.
[8] Bhattacharya, B. B. and Mukherjee, S. (2018). Inference in Ising models. Bernoulli24 493–525. · Zbl 1408.62033
[9] Burnašev, M. V. (1979). Minimax detection of an imperfectly known signal against a background of Gaussian white noise. Teor. Veroyatn. Primen.24 106–118.
[10] Cai, T. T. and Yuan, M. (2014). Rate-optimal detection of very short signal segments. Preprint. Available at arXiv:1407.2812.
[11] Chatterjee, S. (2005). Concentration Inequalities with Exchangeable Pairs. Ph.D. thesis, Stanford University. Available at arXiv:math/0507526.
[12] Chatterjee, S. (2007a). Estimation in spin glasses: A first step. Ann. Statist.35 1931–1946. · Zbl 1126.62128
[13] Chatterjee, S. (2007b). Stein’s method for concentration inequalities. Probab. Theory Related Fields138 305–321. · Zbl 1116.60056
[14] Chatterjee, S. and Dey, P. S. (2010). Applications of Stein’s method for concentration inequalities. Ann. Probab.38 2443–2485. · Zbl 1203.60023
[15] Comets, F. and Gidas, B. (1991). Asymptotics of maximum likelihood estimators for the Curie–Weiss model. Ann. Statist.19 557–578. · Zbl 0749.62018
[16] Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist.32 962–994. · Zbl 1092.62051
[17] Ellis, R. S. and Newman, C. M. (1978). The statistics of Curie–Weiss models. J. Stat. Phys.19 149–161.
[18] Grimmett, G. (2006). The Random-Cluster Model. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 333. Springer, Berlin.
[19] Guyon, X. (1995). Random Fields on a Network: Modeling, Statistics, and Applications. Springer, New York. · Zbl 0839.60003
[20] Hall, P. and Jin, J. (2008). Properties of higher criticism under strong dependence. Ann. Statist.36 381–402. · Zbl 1139.62049
[21] Hall, P. and Jin, J. (2010). Innovated higher criticism for detecting sparse signals in correlated noise. Ann. Statist.38 1686–1732. · Zbl 1189.62080
[22] Ingster, Y. I. (1994). Minimax detection of a signal in \(l_{p}\) metrics. J. Math. Sci.68 503–515. · Zbl 0836.94010
[23] Ingster, Y. I. (1998). Minimax detection of a signal for \(l^{n}\)-balls. Math. Methods Statist.7 401–428. · Zbl 1103.62312
[24] Ingster, Y. I. and Suslina, I. A. (2003). Nonparametric Goodness-of-Fit Testing Under Gaussian Models. Lecture Notes in Statistics169. Springer, New York. · Zbl 1013.62049
[25] Ingster, Y. I., Tsybakov, A. B. and Verzelen, N. (2010). Detection boundary in sparse regression. Electron. J. Stat.4 1476–1526. · Zbl 1329.62314
[26] Ising, E. (1925). Beitrag zur theorie des ferromagnetismus. Zeitschrift Für Physik A Hadrons and Nuclei31 253–258.
[27] Jin, J. and Ke, Z. T. (2016). Rare and weak effects in large-scale inference: Methods and phase diagrams. Statist. Sinica26 1–34. · Zbl 1419.62152
[28] Kac, M. (1959). On the Partition Function of a One-Dimensional Gas. Phys. Fluids2 8–12. · Zbl 0085.44603
[29] Majewski, J., Li, H. and Ott, J. (2001). The Ising model in physics and statistical genetics. Am. J. Hum. Genet.69 853–862.
[30] Mézard, M. and Montanari, A. (2009). Information, Physics, and Computation. Oxford Univ. Press, Oxford.
[31] Mukherjee, S. (2013). Consistent estimation in the two star exponential random graph model. Preprint. Available at arXiv:1310.4526.
[32] Mukherjee, R., Mukherjee, S. and Yuan, M. (2018). Supplement to “Global testing against sparse alternatives under Ising models.” DOI:10.1214/17-AOS1612SUPP.
[33] Mukherjee, R., Pillai, N. S. and Lin, X. (2015). Hypothesis testing for high-dimensional sparse binary regression. Ann. Statist.43 352–381. · Zbl 1308.62094
[34] Nishimori, H. (2001). Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford Univ. Press, New York. · Zbl 1103.82002
[35] Onsager, L. (1944). Crystal statistics. I. A two-dimensional model with an order-disorder transition. Phys. Rep.65 117–149. · Zbl 0060.46001
[36] Park, J. and Newman, M. E. J. (2004). Solution of the two-star model of a network. Phys. Rev. E (3) 70 066146.
[37] Stauffer, D. (2008). Social applications of two-dimensional Ising models. Am. J. Phys.76 470–473.
[38] Wu, Z., Sun, Y., He, S., Cho, J., Zhao, H. and Jin, J. (2014). Detection boundary and higher criticism approach for rare and weak genetic effects. Ann. Appl. Stat.8 824–851. · Zbl 1454.62420
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.