×

Spherical mesh adaptive direct search for separating quasi-uncorrelated sources by range-based independent component analysis. (English) Zbl 1418.62248

Summary: It is seemingly paradoxical to the classical definition of the independent component analysis (ICA), that in reality, the true sources are often not strictly uncorrelated. With this in mind, this letter concerns a framework to extract quasi-uncorrelated sources with finite supports by optimizing a range-based contrast function under unit-norm constraints (to handle the inherent scaling indeterminacy of ICA) but without orthogonality constraints. Albeit the appealing contrast properties of the range-based function (e.g., the absence of mixing local optima), the function is not differentiable everywhere. Unfortunately, there is a dearth of literature on derivative-free optimizers that effectively handle such a nonsmooth yet promising contrast function. This is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit-norm columns with the following objectives: to ascertain convergence to a Clarke stationary point of the contrast function and adhere to the necessary unit-norm constraints more naturally. The proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search (MADS) method to handle locally Lipschitz objective functions defined on the sphere. The applicability of the algorithm in the ICA domain is demonstrated with simulations involving natural, face, aerial, and texture images.

MSC:

62H25 Factor analysis and principal components; correspondence analysis
62F35 Robustness and adaptive procedures (parametric inference)
68T05 Learning and adaptive systems in artificial intelligence

Software:

FAME; BSDS
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Absil, P.-A., & Gallivan, K. A. (2006). Joint diagonalization on the oblique manifold for independent component analysis. In Proceedings of the 31st IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. 5, pp. 945-948). Toulouse, France: IEEE. ,
[2] Absil, P.-A., Mahony, R., & Sepulchre, R. (2008). Optimization algorithms on matrix manifolds. Princeton, NJ: Princeton University Press. , · Zbl 1147.65043
[3] Audet, C., & Dennis, J. E., Jr. (2002). Analysis of generalized pattern searches. SIAM Journal on Optimization, 13(3), 889-903. , · Zbl 1053.90118
[4] Audet, C., & Dennis, J. E., Jr. (2006). Mesh adaptive direct search algorithms for constrained optimization. SIAM Journal on Optimization, 17(1), 188-217. , · Zbl 1112.90078
[5] Boscolo, R., Pan, H., & Roychowdhury, V. P. (2004). Independent component analysis based on nonparametric density estimation. IEEE Transactions on Neural Networks, 15(1), 55-65. ,
[6] Cardoso, J.-F., & Souloumiac, A. (1993). Blind beamforming for non gaussian signals. IEE Proceedings-F: Radar and Signal Processing, 140(6), 362-370. ,
[7] Coope, I. D., & Price, C. J. (2000). Frame based methods for unconstrained optimization. Journal of Optimization Theory and Applications, 107(2), 261-274. , · Zbl 0983.90074
[8] Douglas, S. C., Amari, S.-i., & Kung, S.-Y. (2000). On gradient adaptation with unit-norm constraints. IEEE Transactions on Signal Processing, 48(6), 1843-1847. ,
[9] Hyvärinen, A. (1999a). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626-634. ,
[10] Hyvärinen, A. (1999b). Survey on independent component analysis. Neural Computing Surveys, 2(4), 94-128.
[11] Hyvärinen, A. (2011). Independent component analysis: Recent advances. Philosophical Transactions of the Royal Society A, 1, 1-25.
[12] Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley-Interscience. ,
[13] Karhunen, J., Oja, E., Wang, L., Vigário, R., & Joutsensalo, J. (1997). A class of neural networks for independent component analysis. IEEE Transactions on Neural Networks, 8(3), 486-504. ,
[14] Lee, J. A., Vrins, F., & Verleysen, M. (2006a, September). A least absolute bound approach to ICA-application to the MLSP 2006 competition. In Proceedings of the 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing (pp. 41-46). Maynooth, Ireland: IEEE. ,
[15] Lee, J. A., Vrins, F., & Verleysen, M. (2006b, April). Non-orthogonal support-width ICA. In Proceedings of the 14th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (pp. 351-358). Bruges, Belgium: ESANN.
[16] Li, X.-L., & Adalı, T. (2010). Independent component analysis by entropy bound minimization. IEEE Transactions on Signal Processing, 58(10), 5151-5164. , · Zbl 1392.94305
[17] Makeig, S., Bell, A. J., Jung, T.-P., & Sejnowski, T. J. (1996). Independent component analysis of electroencephalographic data. In D. Touretzky, M. Moser, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 145-151). Cambridge, MA: MIT Press.
[18] Martin, D. R., Fowlkes, C., Tal, D., & Malik, J. (2001, July). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the 8th IEEE International Conference on Computer Vision (Vol. 2, pp. 416-425). Vancouver, Canada: IEEE. ,
[19] Pham, D.-T. (2000). Blind separation of instantaneous mixture of sources based on order statistics. IEEE Transactions on Signal Processing, 48(2), 363-375. , · Zbl 1009.94536
[20] Pham, D.-T. (2001). Joint approximate diagonalization of positive definite Hermitian matrices. SIAM Journal on Matrix Analysis and Applications, 22(4), 1136-1152. , · Zbl 1008.65020
[21] Pham, D.-T., & Vrins, F. (2006). Discriminacy of the minimum range approach to the simultaneous blind separation of bounded sources. In M. Verleysen (Ed.), Advances in Computational Intelligence and Learning (pp. 377-382). Bruges, Belgium: ESANN.
[22] Selvan, S. E., Amato, U., Qi, C., Gallivan, K. A., Carfora, M. F., Larobina, M., (2012). Descent algorithms on oblique manifold for source-adaptive ICA contrast. IEEE Transactions on Neural Networks and Learning Systems, 23(12), 1930-1947. ,
[23] Selvan, S. E., Chattopadhyay, A., Amato, U., & Absil, P.-A. (2012, April). Range-based non-orthogonal ICA using cross-entropy method. In Proceedings of the 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (pp. 519-524). Bruges, Belgium: ESANN.
[24] Seth, S., Rao, M., Park, I., & Príncipe, J. C. (2011). A unified framework for quadratic measures of independence. IEEE Transactions on Signal Processing, 59(8), 3624-3635. , · Zbl 1392.94698
[25] Shen, H., Diepold, K., & Hüper, K. (2009, August-September). Geometric algorithms for the non-whitened one-unit linear independent component analysis problem. In Proceedings of the IEEE/SP 15th Workshop on Statistical Signal Processing (pp. 381-384). Cardiff, UK: IEEE. ,
[26] Shen, H., & Hüper, K. (2009, April). Block Jacobi-type methods for non-orthogonal joint diagonalisation. In Proceedings of the 34th IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 3285-3288). Taipei, Taiwan: IEEE. ,
[27] Shen, H., & Kleinsteuber, M. (2010). Complex blind source separation via simultaneous strong uncorrelating transform. Latent Variable Analysis and Signal Separation, 6365, 287-294. ,
[28] Stegmann, M. B., Ersbøll, B. K., & Larsen, R. (2003). FAME—A flexible appearance modelling environment. IEEE Transactions on Medical Imaging, 22(10), 1319-1331. ,
[29] Torczon, V. (1997). On the convergence of pattern search algorithms. SIAM Journal on Optimization, 7(1), 1-25. , · Zbl 0884.65053
[30] Trendafilov, N. T., & Lippert, R. A. (2002). The multimode Procrustes problem. Linear Algebra and Its Applications, 349(1), 245-264. , · Zbl 0999.65051
[31] van der Veen, A.-J. (2001, May). Joint diagonalization via subspace fitting techniques. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. 5, pp. 2773-2776). Salt Lake City, UT: IEEE. ,
[32] Vrins, F. (2007). Contrast properties of entropic criteria for blind source separation: A unifying framework based on information-theoretic inequalities. Unpublished doctoral dissertation, Faculté des Sciences Appliquées, Université catholique de Louvain.
[33] Vrins, F., Lee, J. A., & Verleysen, M. (2007). A minimum-range approach to blind extraction of bounded sources. IEEE Transactions on Neural Networks, 18(3), 809-822. ,
[34] Yeredor, A. (2002). Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation. IEEE Transactions on Signal Processing, 50(7), 1545-1553. , · Zbl 1369.15005
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.