×

Realization problem of multi-layer cellular neural networks. (English) Zbl 1400.34061

Summary: This paper investigates whether the output space of a multi-layer cellular neural network can be realized via a single layer cellular neural network in the sense of the existence of finite-to-one map from one output space to the other. Whenever such realization exists, the phenomena exhibited in the output space of the revealed single layer cellular neural network is at most a constant multiple of the phenomena exhibited in the output space of the original multi-layer cellular neural network. Meanwhile, the computation complexity of a single layer system is much less than the complexity of a multi-layer system. Namely, one can trade the precision of the results for the execution time. We remark that a routine extension of the proposed methodology in this paper can be applied to the substitution of hidden spaces although the detailed illustration is omitted.

MSC:

34C20 Transformation and reduction of ordinary differential equations and systems, normal forms
37B40 Topological entropy
92B20 Neural networks for/in biological studies, artificial life and related topics

Software:

darch; CNN
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Arena, P.; Baglio, S.; Fortuna, L.; Manganaro, G., Self-organization in a two-layer CNN, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 45, 157-162, (1998)
[2] Ban, J.-C.; Chang, C.-H., On the monotonicity of entropy for multi-layer cellular neural networks, International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 19, 3657-3670, (2009) · Zbl 1182.37016
[3] Ban, J.-C.; Chang, C.-H., The learning problem of multi-layer neural networks, Neural Networks, 46, 116-123, (2013) · Zbl 1296.68121
[4] Ban, J.-C.; Chang, C.-H.; Lin, S.-S.; Lin, Y.-H., Spatial complexity in multi-layer cellular neural networks, Journal of Differential Equations, 246, 552-580, (2009) · Zbl 1153.37005
[5] Bengio, Y., Learning deep architectures for AI, Foundations and Trends in Machine Learning, 2, 1-127, (2009) · Zbl 1192.68503
[6] Bengio, Y.; LeCun, Y., Scaling learning algorithms towards AI, (Bottou, L.; Chapelle, O.; DeCoste, D.; Weston, J., Large scale kernel machines, (2007), MIT Press)
[7] Carmona, R., Jimenez-Garrido, F., Dominguez-Castro, R., Espejo, S., & Rodriguez-Vazquez, A. (2002). CMOS realization of a 2-layer cnn universal machine chip. In Proceedings of the 2002 7th IEEE international workshop on cellular neural networks and their applications, 2002. CNNA 2002 (pp. 444-451).
[8] Chang, C.-H., Deep and shallow architecture of multi-layer neural networks, IEEE Transactions on Neural Networks and Learning Systems, (2015), (published online)
[9] Chua, L. O., (CNN: a paradigm for complexity, World Scientific Series on Nonlinear Science, Series A, vol. 31, (1998), World Scietific Singapore) · Zbl 0916.68132
[10] Chua, L. O.; Roska, T., Cellular neural networks and visual computing, (2002), Cambridge University Press
[11] Chua, L. O.; Shi, B. E., Multiple layer cellular neural networks: A tutorial, (Deprettere, E. F.; van der Veen, A.-J., Algorithms and parallel VLSI architectures, (1991), Elsevier Amsterdam, The Netherlands), 137-168 · Zbl 0800.68591
[12] Chua, L. O.; Yang, L., Cellular neural networks: applications, IEEE Transactions on Circuits and Systems, 35, 1273-1290, (1988)
[13] Chua, L. O.; Yang, L., Cellular neural networks: theory, IEEE Transactions on Circuits and Systems, 35, 1257-1272, (1988) · Zbl 0663.94022
[14] Coven, E. M.; Paul, M., Sofic systems, Israel Journal of Mathematics, 20, 165-177, (1975) · Zbl 0307.28015
[15] Crounse, K. R.; Chua, L. O., Methods for image processing and pattern formation in cellular neural networks: A tutorial, IEEE Transactions on Circuits and Systems, 42, 583-601, (1995)
[16] Crounse, K. R.; Roska, T.; Chua, L. O., Image halftoning with cellular neural networks, IEEE Transactions on Circuits and Systems, 40, 267-283, (1993) · Zbl 0800.94318
[17] Fukushima, K., Artificial vision by multi-layered neural networks: neocognitron and its advances, Neural Networks, 37, 103-119, (2013)
[18] Fukushima, K., Training multi-layered neural network neocognitron, Neural Networks, 40, 18-31, (2013) · Zbl 1283.68313
[19] Hinton, G. E.; Osindero, S.; Teh, Y., A fast learning algorithm for deep belief nets, Neural Computation, 18, 1527-1554, (2006) · Zbl 1106.68094
[20] Hsu, C.-H.; Juang, J.; Lin, S.-S.; Lin, W.-W., Cellular neural networks: local patterns for general template, International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 10, 1645-1659, (2000) · Zbl 1090.34561
[21] Li, X., Analysis of complete stability for discrete-time cellular neural networks with piecewise linear output functions, Neural Computation, 21, 1434-1458, (2009) · Zbl 1162.68559
[22] Lin, K.; Simossis, V.; Taylor, W.; Heringa, J., A simple and fast secondary structure prediction method using hidden neural networks, Bioinformatics, 21, 152-159, (2005)
[23] Mattick, J., Challenging the dogma: the hidden layer of non-protein-coding rnas in complex organisms, Bioessays, 25, 930-939, (2003)
[24] Murugesh, V., Image processing applications via time-multiplexing cellular neural network simulator with numerical integration algorithms, International Journal of Computer Mathematics, 87, 840-848, (2010) · Zbl 1191.94020
[25] Peng, Jun; Zhang, Du; Liao, Xiaofeng, A digital image encryption algorithm based on hyper-chaotic cellular neural network, Fundamenta Informaticae, 90, 269-282, (2009)
[26] Tarca, A.; Carey, V.; Chen, X.-W.; Romero, R.; Drăghici, S., Machine learning and its applications to biology, PLOS Computational Biology, 3, 953-963, (2007)
[27] Török, L.; Roska, T., Stability of multi-layer cellular neural/nonlinear networks, International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 14, 3567-3586, (2004) · Zbl 1129.37363
[28] Utgoff, P. E.; Stracuzzi, D. J., Many-layered learning, Neural Computation, 14, 2497-2539, (2002) · Zbl 1018.68069
[29] Williams, R. F., Classification of subshifts of finite type, Annals of Mathematics, 98, 120-153, (1973), Errata, Annals of Math. 99 (1974), 380-381 · Zbl 0282.58008
[30] Xavier-de Souza, S.; Yalcin, M. E.; Suykens, J. A.K.; Vandewalle, J., Toward CNN chip-specific robustness, IEEE Transactions on Circuits and Systems. I. Regular Papers, 51, 892-902, (2004)
[31] Yang, Z., Nishio, Y., & Ushida, A. (2001). A two layer CNN in image processing applications. In Proc. of the 2001 international symposium on nonlinear theory and its applications (pp. 67-70).
[32] Yang, Z.; Nishio, Y.; Ushida, A., Image processing of two-layer CNNs—applications and their stability, IEICE Transactions on Fundamentals, E85-A, 2052-2060, (2002)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.