×

A smoothing interval neural network. (English) Zbl 1257.68130

Summary: In many applications, it is natural to use interval data to describe various kinds of uncertainties. This paper is concerned with an interval neural network with a hidden layer. For the original interval neural network, it might cause oscillation in the learning procedure as indicated in our numerical experiments. In this paper, a smoothing interval neural network is proposed to prevent the weights oscillation during the learning procedure. Here, by smoothing we mean that, in a neighborhood of the origin, we replace the absolute values of the weights by a smooth function of the weights in the hidden layer and output layer. The convergence of a gradient algorithm for training the smoothing interval neural network is proved. Supporting numerical experiments are provided.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
92B20 Neural networks for/in biological studies, artificial life and related topics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, UK, 1995. · Zbl 1194.81012
[2] M. Perez, “Artificial neural networks and bankruptcy forecasting: a state of the art,” Neural Computing and Applications, vol. 15, no. 2, pp. 154-163, 2006. · Zbl 05027635
[3] G. Zhang, B. E. Patuwo, and M. Y. Hu, “Forecasting with artificial neural networks: the state of the art,” International Journal of Forecasting, vol. 14, no. 1, pp. 35-62, 1998.
[4] M. W. Craven and J. W. Shavlik, “Using neural networks for data mining,” Future Generation Computer Systems, vol. 13, no. 2-3, pp. 211-229, 1997.
[5] S. K. Pal, V. Talwar, and P. Mitra, “Web mining in soft computing framework: relevance, state of the art and future directions,” IEEE Transactions on Neural Networks, vol. 13, no. 5, pp. 1163-1177, 2002.
[6] K. I. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Networks, vol. 2, no. 3, pp. 183-192, 1989.
[7] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks, vol. 4, no. 2, pp. 251-257, 1991.
[8] H. White, “Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings,” Neural Networks, vol. 3, no. 5, pp. 535-549, 1990.
[9] H. Ishibuchi and H. Tanaka, “An extension of the BP algorithm to interval input vectors,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN’91), vol. 2, pp. 1588-1593, Singapore, 1991.
[10] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533-536, 1986. · Zbl 1369.68284
[11] D. E. Rumelhart, J. L. McClelland, and The PDP Research Group, Parallel Distributed Processing, vol. 1, MIT Press, Cambridge, Mass, USA, 1986.
[12] C. A. Hernandez, J. Espi, K. Nakayama, and M. Fernandez, “Interval arithmetic backpropagation,” in Proceedings of International Joint Conference on Neural Networks, vol. 1, pp. 375-378, Nagoya, Japan, October 1993.
[13] A. M. S. Roque, C. Maté, J. Arroyo, and A. Sarabia, “IMLP: applying multi-layer perceptrons to interval-valued data,” Neural Processing Letters, vol. 25, no. 2, pp. 157-169, 2007. · Zbl 05193885
[14] H. M. Shao and G. F. Zheng, “Convergence analysis of a back-propagation algorithm with adaptive momentum,” Neurocomputing, vol. 74, no. 5, pp. 749-752, 2011. · Zbl 05849798
[15] W. Wu, J. Wang, M. S. Cheng, and Z. X. Li, “Convergence analysis of online gradient method for BP neural networks,” Neural Networks, vol. 24, no. 1, pp. 91-98, 2011. · Zbl 1217.68191
[16] D. P. Xu, H. S. Zhang, and L. J. Liu, “Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural networks,” Neural Computation, vol. 22, no. 10, pp. 2655-2677, 2010. · Zbl 1208.68188
[17] J. Wang, J. Yang, and W. Wu, “Convergence of cyclic and almost-cyclic learning with momentum for feedforward neural networks,” IEEE Transactions on Neural Networks, vol. 22, no. 8, pp. 1297-1306, 2011.
[18] R. E. Moore, Interval Analysis, Prentice-Hall, Englewood Cliffs, NJ, USA, 1966. · Zbl 0176.13301
[19] T. Sunaga, “Theory of an interval algebra and its applications to numerical analysis,” RAAG Memoirs, vol. 2, pp. 29-46, 1958. · Zbl 0176.13201
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.