×

Blind nonnegative source separation using biological neural networks. (English) Zbl 1414.92040

Summary: Blind source separation – the extraction of independent sources from a mixture – is an important problem for both artificial and natural signal processing. Here, we address a special case of this problem when sources (but not the mixing matrix) are known to be nonnegative – for example, due to the physical nature of the sources. We search for the solution to this problem that can be implemented using biologically plausible neural networks. Specifically, we consider the online setting where the data set is streamed to a neural network. The novelty of our approach is that we formulate blind nonnegative source separation as a similarity matching problem and derive neural networks from the similarity matching objective. Importantly, synaptic weights in our networks are updated according to biologically plausible local learning rules.

MSC:

92B20 Neural networks for/in biological studies, artificial life and related topics

Software:

SymNMF
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757-763). Cambridge, MA: MIT Press.
[2] Asari, H., Pearlmutter, B. A., & Zador, A. M. (2006). Sparse representations for the cocktail party problem. Journal of Neuroscience, 26(28), 7477-7490. ,
[3] Bee, M. A., & Micheyl, C. (2008). The cocktail party problem: What is it? How can it be solved? And why should animal behaviorists study it?Journal of Comparative Psychology, 122(3), 235. ,
[4] Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159. ,
[5] Bliss, T. V., & Gardner-Medwin, A. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the unanaesthetized rabbit following stimulation of the perforant path. Journal of Physiology, 232(2), 357. ,
[6] Bliss, T. V., & Lømo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. Journal of Physiology, 232(2), 331-356. ,
[7] Bronkhorst, A. W. (2000). The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions. Acta Acustica United with Acustica, 86(1), 117-128.
[8] Bull, D. R. (2014). Communicating pictures: A course in image and video coding. Orlando, FL: Academic Press. ,
[9] Comon, P. (1994). Independent component analysis, a new concept?Signal Processing, 36(3), 287-314. , · Zbl 0791.62004
[10] Comon, P., & Jutten, C. (2010). Handbook of blind source separation: Independent component analysis and applications. Orlando, FL: Academic Press.
[11] David, S. V., Vinje, W. E., & Gallant, J. L. (2004). Natural stimulus statistics alter the receptive field structure of V1 neurons. Journal of Neuroscience, 24(31), 6991-7006. ,
[12] Donoho, D., & Stodden, V. (2003). When does non-negative matrix factorization give a correct decomposition into parts? In S. Thrun, K. Saul, & P. B. Schölkopf (Eds.), Advances in neural information processing systems, 16. Cambridge, MA: MIT Press.
[13] Eagleman, D. M., Coenen, O. J.-M. D., Mitsner, V., Bartol, T. M., Bell, A. J., and Sejnowski, T. J. (2001). Cerebellar glomeruli: Does limited extracellular calcium implement a sparse encoding strategy? In Proceedings of the 8th Annual Joint Symposium on Neural Computation.
[14] Golumbic, E. M. Z., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., … Poeppel, D. (2013). Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party.”Neuron, 77(5), 980-991. ,
[15] Hu, T., Pehlevan, C., & Chklovskii, D. B. (2014). A Hebbian/anti-Hebbian network for online sparse dictionary learning derived from symmetric matrix factorization. In Proceedings of the Asilomar Conference on Signals, Systems and Computers (pp. 613-619). Piscataway, NJ: IEEE. ,
[16] Huang, K., Sidiropoulos, N., & Swami, A. (2014). Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition. IEEE Transactions on Signal Processing, 62(1), 211-224. , · Zbl 1393.15016
[17] Hyvarinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626-634. ,
[18] Hyvärinen, A., & Hoyer, P. (2000). Emergence of phase-and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12(7), 1705-1720. ,
[19] Hyvärinen, A., & Oja, E. (1997). A fast fixed-point algorithm for independent component analysis. Neural Computation, 9(7), 1483-1492. ,
[20] Hyvärinen, A., & Oja, E. (1998). Independent component analysis by general nonlinear Hebbian-like learning rules. Signal Processing, 64(3), 301-313. , · Zbl 0893.94034
[21] Hyvärinen, A., & Oja, E. (2000). Independent component analysis: Algorithms and applications. Neural Networks, 13(4), 411-430. ,
[22] Isomura, T., Kotani, K., & Jimbo, Y. (2015). Cultured cortical neurons can perform blind source separation according to the free-energy principle. PLoS Comput. Biol., 11(12), e1004643. ,
[23] Isomura, T., & Toyoizumi, T. (2016). A local learning rule for independent component analysis. Scientific Reports, 6. , · Zbl 1469.94031
[24] Jutten, C., & Hérault, J. (1991). Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24(1), 1-10. , · Zbl 0729.73650
[25] Komatsu, Y. (1994). Age-dependent long-term potentiation of inhibitory synaptic transmission in rat visual cortex. Journal of Neuroscience, 14(11), 6488-6499. ,
[26] Kuang, D., Park, H., & Ding, C. H. (2012). Symmetric nonnegative matrix factorization for graph clustering. In Proceedings of the 2012 SIAM International Conference on Data Mining (vol. 12, pp. 106-117). Philadelphia: SIAM. ,
[27] Kuang, D., Yun, S., & Park, H. (2015). Symnmf: Nonnegative low-rank approximation of a similarity matrix for graph clustering. Journal of Global Optimization, 62(3), 545-574. , · Zbl 1326.90080
[28] Kullmann, D. M., Moreau, A. W., Bakiri, Y., & Nicholson, E. (2012). Plasticity of inhibition. Neuron, 75(6), 951-962. ,
[29] Laurberg, H., Christensen, M. G., Plumbley, M. D., Hansen, L. K., & Jensen, S. H. (2008). Theorems on positive data: On the uniqueness of NMF. Computational Intelligence and Neuroscience, 2008, 764206. ,
[30] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. ,
[31] Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788-791. , · Zbl 1369.68285
[32] Li, Y., Liang, Y., & Risteski, A. (2016). Recovery guarantee of non-negative matrix factorization via alternating updates. arXiv preprint:1611.03819
[33] Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21(3), 105-117. ,
[34] Linsker, R. (1997). A local learning rule that enables information maximization for arbitrary input distributions. Neural Computation, 9(8), 1661-1665. ,
[35] Maffei, A., Nataraj, K., Nelson, S. B., & Turrigiano, G. G. (2006). Potentiation of cortical inhibition by visual deprivation. Nature, 443(7107), 81-84. ,
[36] Malenka, R. C., & Bear, M. F. (2004). LTP and LTD: An embarrassment of riches. Neuron, 44(1), 5-21. ,
[37] McDermott, J. H. (2009). The cocktail party problem. Current Biology, 19(22), R1024-R1027. ,
[38] Mesgarani, N., & Chang, E. F. (2012). Selective cortical representation of attended speaker in multi-talker speech perception. Nature, 485(7397), 233-236. ,
[39] Narayan, R., Best, V., Ozmeral, E., McClaine, E., Dent, M., Shinn-Cunningham, B., & Sen, K. (2007). Cortical interference effects in the cocktail party problem. Nature Neuroscience, 10(12), 1601-1607. ,
[40] Oja, E. (1982). Simplified neuron model as a principal component analyzer. J. Math. Biol., 15(3), 267-273. , · Zbl 0488.92012
[41] Oja, E., & Plumbley, M. (2004). Blind separation of positive sources by globally convergent gradient search. Neural Computation, 16(9), 1811-1825. , · Zbl 1086.68574
[42] Ouedraogo, W. S. B., Souloumiac, A., & Jutten, C. (2010). Non-negative independent component analysis algorithm based on 2D Givens rotations and a newton optimization. In V. Vigneron, V. Zarzoso, E. Moreau, R. Gribonval, & E. Vincent (Eds.), Latent variable analysis and signal separation (pp. 522-529). New York: Springer. ,
[43] Paatero, P., & Tapper, U. (1994). Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5(2), 111-126. ,
[44] Patel, A. B., Nguyen, M. T., & Baraniuk, R. (2016). A probabilistic framework for deep learning. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in neural information processing systems (pp. 2550-2558). Red Hook, NY: Curran.
[45] Pehlevan, C., & Chklovskii, D. (2014). A Hebbian/anti-Hebbian network derived from online non-negative matrix factorization can cluster and discover sparse features. In Proceedings of the Asilomar Conference on Signals, Systems and Computers (pp. 769-775). Piscataway, NJ: IEEE. ,
[46] Pehlevan, C., & Chklovskii, D. (2015a). A normative theory of adaptive dimensionality reduction in neural networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in neural information processing systems (pp. 2260-2268). Red Hook, NY: Curran.
[47] Pehlevan, C., & Chklovskii, D. (2015b). Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening. In Proceedings of the 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (pp. 1458-1465). Piscataway, NJ: IEEE. ,
[48] Pehlevan, C., Hu, T., & Chklovskii, D. (2015). A Hebbian/anti-Hebbian neural network for linear subspace learning: A derivation from multidimensional scaling of streaming data. Neural Computation, 27, 1461-1495. , · Zbl 1472.68153
[49] Plumbley, M. (1994). A subspace network that determines its own output dimension (Tech. Report). London: Kings College London.
[50] Plumbley, M. (1996). Information processing in negative feedback neural networks. Network-Comp. Neural, 7(2), 301-305. , · Zbl 0895.94008
[51] Plumbley, M. D. (2001). Adaptive lateral inhibition for non-negative ICA. In Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation (pp. 516-521). New York: Springer.
[52] Plumbley, M. (2002). Conditions for nonnegative independent component analysis. IEEE Signal Processing Letters, 9(6), 177-180. ,
[53] Plumbley, M. D. (2003). Algorithms for nonnegative independent component analysis. IEEE Transactions on Neural Networks, 14(3), 534-543. ,
[54] Plumbley, M. D., & Oja, E. (2004). A “nonnegative PCA” algorithm for independent component analysis. IEEE Transactions on Neural Networks, 15(1), 66-76. ,
[55] Wright, S. J. (2015). Coordinate descent algorithms. Mathematical Programming, 151(1), 3-34. , · Zbl 1317.49038
[56] Yuan, Z., & Oja, E. (2004). A fastica algorithm for non-negative independent component analysis. In C. G. Puntonet & A. Prieto (Eds.), Independent component analysis and blind signal separation (pp. 1-8). New York: Springer. ,
[57] Zheng, C.-H., Huang, D.-S., Sun, Z.-L., Lyu, M. R., & Lok, T.-M. (2006). Nonnegative independent component analysis based on minimizing mutual information technique. Neurocomputing, 69(7), 878-883. ,
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.