Efficient independent component analysis. (English) Zbl 1114.62033

Chen, Aiyou; Bickel, Peter J.

Ann. Stat. 34, No. 6, 2825-2855 (2006).

Summary: Independent component analysis (ICA) has been widely used for blind source separation in many fields, such as brain imaging analysis, signal processing and telecommunication. Many statistical techniques based on \(M\)-estimates have been proposed for estimating the mixing matrix.
Recently, several nonparametric methods have been developed, but in-depth analysis of asymptotic efficiency has not been available. We analyze ICA using semiparametric theories and propose a straightforward estimate based on the efficient score function by using B-spline approximations. The estimate is asymptotically efficient under moderate conditions and exhibits better performance than standard ICA methods in a variety of simulations.

MSC:

62G05	Nonparametric estimation
62H12	Estimation in multivariate analysis
62G20	Asymptotic properties of nonparametric inference
65C60	Computational problems in statistics (MSC2010)

Keywords:

semiparametric models; efficient score functions; asymptotically efficient; generalized \(M\)-estimators; B-splines

Software:

FastICA

References:

[1]	Amari, S. (2002). Independent component analysis and method of estimating functions. IEICE Trans. Fundamentals Electronics, Communications and Computer Sciences E85-A 540–547.
[2]	Amari, S. and Cardoso, J. (1997). Blind source separation—semiparametric statistical approach. IEEE Trans. Signal Process. 45 2692–2700.
[3]	Bach, F. and Jordan, M. (2002). Kernel independent component analysis. J. Mach. Learn. Res. 3 1–48. · Zbl 1088.68689 · doi:10.1162/153244303768966085
[4]	Bickel, P. and Doksum, K. (2001). Mathematical Statistics : Basic Ideas and Selected Topics 1 , 2nd ed. Prentice-Hall, Upper Saddle River, NJ. · Zbl 0403.62001
[5]	Bickel, P., Klaassen, C., Ritov, Y. and Wellner, J. (1998). Efficient and Adaptive Estimation for Semiparametric Models . Springer, New York. · Zbl 0894.62005
[6]	Cardoso, J.-F. (1998). Blind signal separation: Statistical principles. Proc. IEEE 86 2009–2025.
[7]	Cardoso, J.-F. (1999). High-order contrasts for independent component analysis. Neural Comput. 11 157–192.
[8]	Chen, A. and Bickel, P. J. (2005). Consistent independent component analysis and prewhitening. IEEE Trans. Signal Process. 53 3625–3632. · Zbl 1373.62292 · doi:10.1109/TSP.2005.855098
[9]	Comon, P. (1994). Independent component analysis. A new concept? Signal Process. 36 287–314. · Zbl 0791.62004 · doi:10.1016/0165-1684(94)90029-9
[10]	Cox, D. D. (1985). A penalty method for nonparametric estimation of the logarithmic derivative of a density function. Ann. Inst. Statist. Math. 37 271–288. · Zbl 0578.62041 · doi:10.1007/BF02481097
[11]	de Boor, C. (2001). A Practical Guide to Splines , rev. ed. Springer. · Zbl 0987.65015
[12]	Eriksson, J. and Koivunen, V. (2003). Characteristic-function based independent component analysis. Signal Process. 83 2195–2208. · Zbl 1145.94320 · doi:10.1016/S0165-1684(03)00162-2
[13]	Hastie, T. and Tibshirani, R. (2002). Independent component analysis through product density estimation. Technical report, Dept. Statistics, Stanford Univ.
[14]	Huber, P. J. (1985). Projection pursuit (with discussion). Ann. Statist. 13 435–525. · Zbl 0595.62059 · doi:10.1214/aos/1176349519
[15]	Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Networks 10 626–634.
[16]	Hyvärinen, A., Karhunen, J. and Oja, E. (2001). Independent Component Analysis . Wiley, New York.
[17]	Jin, K. (1992). Empirical smoothing parameter selection in adaptive estimation. Ann. Statist. 20 1844–1874. · Zbl 0774.62036 · doi:10.1214/aos/1176348892
[18]	Kagan, A., Linnik, Y. and Rao, C. R. (1973). Characterization Problems in Mathematical Statistics . Wiley, New York. · Zbl 0271.62002
[19]	Lee, T.-W., Girolami, M. and Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-Gaussian and super-Gaussian sources. Neural Comput. 11 417–441.
[20]	Murphy, S. and van der Vaart, A. (2000). On profile likelihood (with discussion). J. Amer. Statist. Assoc. 95 449–485. JSTOR: · Zbl 0995.62033 · doi:10.2307/2669386
[21]	Pham, D. T. and Garat, P. (1997). Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Signal Process. 45 1712–1725. · Zbl 0879.94007 · doi:10.1109/78.599941
[22]	Samarov, A. and Tsybakov, A. (2004). Nonparametric independent component analysis. Bernoulli 10 565–582. · Zbl 1055.62037 · doi:10.3150/bj/1093265630
[23]	Truong, Y. K., Kooperberg, C. and Stone, C. J. (2005). Statistical Modeling with Spline Functions: Methodology and Theory . Springer, New York.
[24]	van de Geer, S. (2000). Applications of Empirical Process Theory . Cambridge Univ. Press. · Zbl 0953.62049
[25]	Vigário, R., Jousmáki, V., Hämäläinen, M., Hari, R. and Oja, E. (1998). Independent component analysis for identification of artifacts in magnetoencephalographic recordings. In Advances in Neural Information Processing Systems 10 229–235. MIT Press, Cambridge.
[26]	Vlassis, N. and Motomura, Y. (2001). Efficient source adaptivity in independent component analysis. IEEE Trans. Neural Networks 12 559–566.