Polynomial-based radial basis function neural networks (P-RBF NNs) realized with the aid of particle swarm optimization. (English) Zbl 1210.68084

Oh, Sung-Kwun; Kim, Wook-Dong; Pedrycz, Witold; Park, Byoung-Jun

Fuzzy Sets Syst. 163, No. 1, 54-77 (2011).

Summary: We design polynomial-based radial basis function neural networks (P-RBF NNs) based on a fuzzy inference mechanism. The essential design parameters (including learning rate, momentum coefficient and fuzzification coefficient of the underlying clustering method) are optimized by means of the particle swarm optimization. The proposed P-RBF NNs dwell upon structural findings about training data that are expressed in terms of a partition matrix resulting from fuzzy clustering in this case being the fuzzy C-means (FCM). The network is of functional nature as the weights between the hidden layer and the output are some polynomials. The use of the polynomial weights becomes essential in capturing the nonlinear nature of data encountered in regression or classification problems. From the perspective of linguistic interpretation, the proposed network can be expressed as a collection of “if-then” fuzzy rules. The architecture of the networks discussed here embraces three functional modules reflecting the three phases of input-output mapping realized in rule-based architectures, namely condition formation, conclusion creation, and aggregation. The proposed classifier is applied to some synthetic and machine learning datasets, and its results are compared with those reported in the previous studies.

MSC:

68T05	Learning and adaptive systems in artificial intelligence
68T10	Pattern recognition, speech recognition

Keywords:

polynomial neural networks; radial basis function neural networks; pattern classification; fuzzy clustering; particle swarm optimization

Software:

EGO; C4.5

References:

[1]	Lawrence, S.; Giles, C.L.; Tsoi, A.C.; Back, A.D., Face recognition: a convolutional neural-network approach, IEEE transactions on neural on networks, 8, 98-113, (1997)
[2]	Lin, S.-H.; Kung, S.-Y.; Lin, L.-J., Face recognition/detection by probabilistic decision-based neural network, IEEE transactions on neural on networks, 8, 114-132, (1997)
[3]	Lippman, R.P., An introduction to computing with neural nets, IEEE ASSP magazine, 4, 2, 4-22, (1981)
[4]	Song, H.H.; Lee, S.W., A self-organizing neural tree for large-set pattern classification, IEEE transactions on neural networks, 9, 369-380, (1998)
[5]	Patrikar, A.; Provence, J., Pattern classification using polynomial networks, Electronics letters, 28, 12, 1109-1110, (1992)
[6]	Cho, K.B.; Wang, B.H., Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction, Fuzzy sets and systems, 83, 325-339, (1996)
[7]	Er, M.J.; Wu, S.Q.; Lu, J.W.; Toh, H.L., Face recognition with radical basis function (RBF) neural networks, IEEE transactions on neural networks, 13, 3, 697-710, (2002)
[8]	Mali, K.; Mitra, S., Symbolic classification, clustering and fuzzy radial basis function network, Fuzzy sets and systems, 152, 553-564, (2005) · Zbl 1101.68775
[9]	Park, B.-J.; Oh, S.-K.; Kim, H.-K., Design of polynomial neural network classifier for pattern classification with two classes, Journal of electrical engineering & technology, 3, 1, 108-114, (2008)
[10]	Kennedy, J., The particle swarm: social adaptation of knowledge, (), 303-308
[11]	Kennedy, J.; Eberhart, R., Particle swarm optimization, (), 1942-1948
[12]	Abido, M.A., Optimal design of power-system stabilizers using particle swarm optimization, IEEE transactions on energy conversion, 17, 3, 406-413, (2002)
[13]	Staiano, A.; Tagliaferri, R.; Pedrycz, W., Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering, Neurocomputing, 69, 1570-1581, (2006)
[14]	Bezdek, J.C., Pattern recognition with fuzzy objective function algorithms, (1981), Plenum Press New York · Zbl 0503.68069
[15]	Oh, S.-K.; Pderycz, W.; Park, B.-J., Self-organizing neurofuzzy networks in modeling software data, Fuzzy sets and systems, 145, 165-181, (2004)
[16]	Gaing, Z.L., A particle swarm optimization approach for optimum design of PID controller in AVR system, IEEE transactions on energy conversion, 19, 2, 384-391, (2004)
[17]	Juang, C.-F.; Lo, C., Zero-order TSK-type fuzzy system learning using a two-phase swarm intelligence algorithm, Fuzzy sets and systems, 159, 2910-2926, (2008)
[18]	Parsopoulos, K.E.; Vrahatis, M.N., On the computation of all global minimizers through particle swarm optimization, IEEE transactions on evolutionary computation, 8, 3, 211-224, (2004)
[19]	Koufakou, A.; Georgiopoulos, M.; Anagnostopoulos, G.; Kasparis, T., Cross-validation in fuzzy ARTMAP for large databases, Neural networks, 14, 9, 1279-1291, (2001)
[20]	Oh, S.-K.; Pedrycz, W.; Roh, S.-B., Hybrid fuzzy set-based polynomial networks and their development with the aid of genetic optimization and information granulation, Applied soft computing, 9, 3, 1068-1089, (2009)
[21]	Hassoun, M.H., Fundamentals of artificial neural networks, (1995), The MIT Press Cambridge, MA · Zbl 0850.68271
[22]	Klawonn, F.; Nauck, D.; Kruse, R., Generating rules from data by fuzzy and neuro-fuzzy methods, (), 223-230
[23]	Quinlan, J.R., Improved use of continuous attributes in C4.5, Journal of artificial intelligence research, 4, 77-90, (1996) · Zbl 0900.68112
[24]	Janikow, C.Z.; Faifer, M., Fuzzy partitioning with FID31, (), 467-471
[25]	Gonçalves, L.B.; Vellasco, M.M.B.R.; Pacheco, M.A.C.; de Souza, F.J., Inverted hierarchical neuro-fuzzy BSP system: a novel neuro-fuzzy model for pattern classification and rule extraction in databases, IEEE transactions on systems, man & cybernetics, part C, 26, 2, (2005)
[26]	Vapnik, V., The nature of statistical learning theory, (March 1995), Springer-Verlag New York
[27]	Jensen, F., An introduction to Bayesian networks, (1996), UCL Press/Springer-Verlag
[28]	R. Adamczak, W. Duch, N. Jankowski, New developments in the Feature Space Mapping model, in: Third Conference on Neural Networks and Their Applications, Kule, October 1997, pp. 65-70.
[29]	Tipping, M.E., The relevance vector machine, Advances in neural information processing systems, 12, November, 652-658, (2000)
[30]	Abbass, H.A., An evolutionary artificial neural networks approach for breast cancer diagnosis, Artificial intelligence in medicine, 25, 3, 265-281, (2002)
[31]	Duda, R.O.; Hart, P.E., Pattern classification and scene analysis, (2002), Wiley New York
[32]	Anagnostopoulos, I.; Maglogiannis, I., Neural network-based diagnostic and prognostic estimations in breast cancer microscopic instances, Medical & biological engineering & computing Springer publishers, 44, 9, 773-784, (2006)
[33]	Young, Z.R., A novel radial basis function neural network for discriminant analysis, IEEE transactions on neural networks, 17, 3, 604-612, (2006)
[34]	Lim, T.; Loh, W.; Shih, Y., A comparison of prediction accuracy, complexity, and training time of thirty three old and new classification algorithms, Machine learning, 40, 203-228, (2000) · Zbl 0969.68669
[35]	S. Aeberhard, D. Coomans, O. de Vel, Comparison of classifiers in high dimensional settings, Department of Computer Science and Department of Mathematics and Statistics, James Cook. University of North Queensland, Technical Report 92-02, 1992. · Zbl 0968.68204
[36]	Kotsiantis, S.B.; Pintelas, P.E., Logitboost of simple Bayesian classifier, Informatica, 29, 53-59, (2005)