A novel kernel for RBF based neural networks. (English) Zbl 1470.68066

Aftab, Wasim; Moinuddin, Muhammad; Shaikh, Muhammad Shafique

Abstr. Appl. Anal. 2014, Article ID 176253, 10 p. (2014).

Summary: Radial basis function (RBF) is well known to provide excellent performance in function approximation and pattern classification. The conventional RBF uses basis functions which rely on distance measures such as Gaussian kernel of Euclidean distance (ED) between feature vector and neuron’s center, and so forth. In this work, we introduce a novel RBF artificial neural network (ANN) where the basis function utilizes a linear combination of ED based Gaussian kernel and a cosine kernel where the cosine kernel computes the angle between feature and center vectors. Novelty of the proposed work relies on the fact that we have shown that there may be scenarios where the two feature vectors (FV) are more prominently distinguishable via the proposed cosine measure as compared to the conventional ED measure. We discuss adaptive symbol detection for multiple phase shift keying (MPSK) signals as a practical example to show where the angle information can be pivotal which in turn justifies our proposed RBF kernel. To corroborate our theoretical developments, we investigate the performance of the proposed RBF for the problems pertaining to three different domains. Our results show that the proposed RBF outperforms the conventional RBF by a remarkable margin.

MSC:

68T05

Learning and adaptive systems in artificial intelligence

Software:

Flavia; Silhouettes

References:

[1]	McCulloch, W. S.; Pitts, W., A logical calculus of the ideas immanent in nervous activity, The Bulletin of Mathematical Biophysics, 5, 4, 115-133 (1943) · Zbl 0063.03860
[2]	Khan, J.; Wei, J. S.; Ringnér, M.; Saal, L. H.; Ladanyi, M.; Westermann, F.; Berthold, F.; Schwab, M.; Antonescu, C. R.; Peterson, C.; Meltzer, P. S., Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks, Nature Medicine, 7, 6, 673-679 (2001)
[3]	Zhang, G.; Patuwo, B. E.; Hu, M. Y., Forecasting with artificial neural networks: the state of the art, International Journal of Forecasting, 14, 1, 35-62 (1998)
[4]	Bertsekas, D. P.; Tsitsiklis, J. N., Neuro-dynamic programming: an overview, Proceedings of the 34th IEEE Conference on Decision and Control
[5]	Haykin, S., Neural Networks: A Comprehensive Foundation (1994), Upper Saddle River, NJ, USA: Prentice Hall PTR, Upper Saddle River, NJ, USA · Zbl 0828.68103
[6]	Fukushima, K., Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biological Cybernetics, 36, 4, 193-202 (1980) · Zbl 0419.92009
[7]	Park, J.; Sandberg, I. W., Universal approximation using radial-basis-function networks, Neural Computation, 3, 2, 246-257 (1991)
[8]	Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain, Psychological Review, 65, 6, 386-408 (1958)
[9]	Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning representations by back-propagating errors, Nature, 323, 6088, 533-536 (1986) · Zbl 1369.68284
[10]	Broomhead, D. S.; Lowe, D., Multivariable functional interpolation and adaptive networks, Complex Systems, 2, 3, 321-355 (1988) · Zbl 0657.68085
[11]	Ackley, D. H.; Hinton, G. E., A learning algorithm for boltzmann machines, Cognitive Science, 9, 1, 147-169 (1985)
[12]	Chen, S.; Cowan, C. F. N.; Grant, P. M., Orthogonal least squares learning algorithm for radial basis function networks, IEEE Transactions on Neural Networks, 2, 2, 302-309 (1991)
[13]	Leung, H.; Lo, T.; Wang, S., Prediction of noisy chaotic time series using an optimal radial basis function neural network, IEEE Transactions on Neural Networks, 12, 5, 1163-1172 (2001)
[14]	Seshagiri, S.; Khalil, H. K., Output feedback control of nonlinear systems using RBF neural networks, IEEE Transactions on Neural Networks, 11, 1, 69-79 (2000)
[15]	Platt, J., A resource-allocating network for function interpolation, Neural Computation, 3, 2, 213-225 (1991)
[16]	Krogh, A.; Vedelsby, J., Neural network ensembles, cross validation, and active learning, Advances in Neural Information Processing Systems, 7, 231-238 (1995), Cambridge, Mass, USA: MIT Press, Cambridge, Mass, USA
[17]	Wettschereck, D.; Dietterich, T., Improving the performance of radial basis function networks by learning center locations, Advances in Neural Information Processing Systems, 4 (1992), San Mateo, Calif, USA: Morgan Kaufmann, San Mateo, Calif, USA
[18]	Ali, S. S. A.; Moinuddin, M.; Raza, K.; Adil, S. H., An adaptive learning rate for RBFNN using time-domain feedback analysis, The Scientific World Journal, 2014 (2014)
[19]	Musavi, M. T.; Ahmed, W.; Chan, K. H.; Faris, K. B.; Hummels, D. M., On the training of radial basis function classifiers, Neural Networks, 5, 4, 595-603 (1992)
[20]	Schilling, R. J.; Carroll, J. J.; Al-Ajlouni, A. F., Approximation of nonlinear systems with radial basis function neural networks, IEEE Transactions on Neural Networks, 12, 1, 1-15 (2001)
[21]	Er, M. J.; Chen, W.; Wu, S., High-speed face recognition based on discrete cosine transform and RBF neural networks, IEEE Transactions on Neural Networks, 16, 3, 679-691 (2005)
[22]	Rudzicz, F., Articulatory knowledge in the recognition of dysarthric speech, IEEE Transactions on Audio, Speech and Language Processing, 19, 4, 947-960 (2011)
[23]	Zeng, Y. J.; Zhuang, J. D., Construction cosine radial basic function neural networks based on artificial immune networks, Advanced Data Mining and Applications. Advanced Data Mining and Applications, Lecture Notes in Computer Science, 6441, 134-141 (2010), Berlin, Germany: Springer, Berlin, Germany
[24]	Zhang, K.; Li, Y.; Scarf, P.; Ball, A., Feature selection for high-dimensional machinery fault diagnosis data using multiple models and radial basis function networks, Neurocomputing, 74, 17, 2941-2952 (2011)
[25]	Acharya, U. R.; Molinari, F.; Sree, S. V.; Chattopadhyay, S.; Ng, K.; Suri, J. S., Automated diagnosis of epileptic EEG using entropies, Biomedical Signal Processing and Control, 7, 4, 401-408 (2012)
[26]	Sridhar, D.; Krishna, M., Brain tumor classification using discrete cosine transform and probabilistic neural network, Proceedings of the International Conference on Signal Processing, Image Processing and Pattern Recognition (ICSIPR ’13)
[27]	Karayiannis, N. B.; Randolph-Gips, M. M., On the construction and training of reformulated radial basis function neural networks, IEEE Transactions on Neural Networks, 14, 4, 835-846 (2003)
[28]	Liu, Z.; Zhao, X.; Zuo, M. J.; Xu, H., Feature selection for fault level diagnosis of planetary gearboxes, Advances in Data Analysis and Classification (2014) · Zbl 1414.93126
[29]	Cho, Y.; Saul, L. K., Kernel methods for deep learning, Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS ’09)
[30]	Karayiannis, N. B.; Xiong, Y., Training reformulated radial basis function neural networks capable of identifying uncertainty in data classification, IEEE Transactions on Neural Networks, 17, 5, 1222-1234 (2006)
[31]	Cover, T. M., Geometrical and Statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE Transactions on Electronic Computers, 14, 3, 326-334 (1965) · Zbl 0192.08403
[32]	Psaltis, D.; Sideris, A.; Yamamura, A. A., A multilayered neural network controller, IEEE Control Systems Magazine, 8, 2, 17-21 (1988)
[33]	Rousseeuw, P. J., Silhouettes: a graphical aid to the interpretation and validation of cluster analysis, Journal of Computational and Applied Mathematics, 20, 53-65 (1987) · Zbl 0636.62059
[34]	Kuncheva, L. I., A theoretical study on six classifier fusion strategies, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 2, 281-286 (2002)
[35]	Mallah, C.; Cope, J.; Orwell, J., Plant leaf classification using probabilistic integration of shape, texture and margin features, Proceedings of the International Conference on Signal Processing, Pattern Recognition and Applications (SPPRA ’13)
[36]	Narendra, K. S.; Parthasarathy, K., Identification and control of dynamical systems using neural networks, IEEE Transactions on Neural Networks, 1, 1, 4-27 (1990)
[37]	Elanayar, V. T. S.; Shin, Y. C., Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems, IEEE Transactions on Neural Networks, 5, 4, 594-603 (1994)
[38]	Lippmann, R. P., Pattern classification using neural networks, IEEE Communications Magazine, 27, 11, 47-50 (1989)
[39]	Er, M. J.; Wu, S.; Lu, J.; Toh, H. L., Face recognition with radial basis function (RBF) neural networks, IEEE Transactions on Neural Networks, 13, 3, 697-710 (2002)