zbMATH — the first resource for mathematics

Using genetic algorithms to select architecture of a feedforward artificial neural network. (English) Zbl 0971.68506
Summary: This paper proposes a model selection methodology for feedforward network models based on the genetic algorithms and makes a number of distinct but inter-related contributions to the model selection literature for the feedforward networks. First, we construct a genetic algorithm which can search for the global optimum of an arbitrary function as the output of a feedforward network model. Second, we allow the genetic algorithm to evolve the type of inputs, the number of hidden units and the connection structure between the inputs and the output layers. Third, we study how introduction of a local elitist procedure which we call the election operator affects the algorithm’s performance. We conduct a Monte Carlo simulation to study the sensitiveness of the global approximation properties of the studied genetic algorithm. Finally, we apply the proposed methodology to the daily foreign exchange returns.

68T05 Learning and adaptive systems in artificial intelligence
Full Text: DOI
[1] Elman, J.L., Finding structure in time, Cognitive sci., 14, 179-211, (1990)
[2] M.I. Jordan, Serial order: a parallel distributed processing approach, UC San Diego, Institute for Cognitive Science Report 8604, 1980.
[3] Gençay, R.; Dechert, W.D., An algorithm for the n Lyapunov exponents of an n-dimensional unknown dynamical system, Physica D, 59, 142-157, (1992) · Zbl 0761.58025
[4] Gençay, R., Nonlinear prediction of noisy time series with feedforward networks, Physics lett. A, 187, 397-403, (1994)
[5] Gençay, R., A statistical framework for testing chaotic dynamics via Lyapunov exponents, Physica D, 89, 261-266, (1996) · Zbl 0886.58052
[6] Dechert, W.D.; Gençay, R., Lyapunov exponents as a nonparametric diagnostic for stability analysis, J. appl. econometrics, 7, 41-60, (1992)
[7] Dechert, W.D.; Gençay, R., The topological invariance of Lyapunov exponents in embedded dynamics, Physica D, 90, 40-55, (1996) · Zbl 0884.34059
[8] Kuan, C.; Liu, T., Forecasting exchange rates using feedforward and recurrent neural networks, J. appl. econometrics, 10, 347-364, (1995)
[9] Swanson, N.; White, H., A model-selection approach to assessing the information in the term structure using linear models and artificial neural networks, J. busi. econ. statist., 13, 265-275, (1995)
[10] Hutchinson, J.M.; Lo, A.W.; Poggio, T., A nonparametric approach to pricing and hedging derivative securities via learning network, J. finance, 3, 851-889, (1994)
[11] Garcia, R.; Gençay, R., Pricing and hedging derivative securities with neural networks and a homogeneity hint, J. econometrics, 94, 93-115, (2000) · Zbl 0942.62130
[12] Holland, J.H., Adaptation in natural and artificial systems, (1975.), The University of Michigan Press Ann Arbor
[13] Michalewicz, Z., Genetic algorithms + data structures = evolution programs, (1996), Springer New York · Zbl 0841.68047
[14] Fogel, D.; Fogel, L.; Porto, V., Evolving neural networks, Biol. cybernet., 63, 487-493, (1990)
[15] Menczer, F.; Parisi, D., Evidence of hyperplanes in the genetic learning of neural networks, Biol. cybernet., 66, 283-289, (1992)
[16] D. Montana, L. Davis, Training feedforward neural networks using genetic algorithms, in: Proceedings of Eleventh International Joint Conference on Artifical Intelligence, N.S. Sridharan (Ed.), Morgan Kaufman Publishers, 1989. · Zbl 0709.68060
[17] Saha, S.; Christensen, J., Genetic design of sparse feedforward neural networks, Inform. sci., 79, 191-200, (1994) · Zbl 0801.93044
[18] Miller, G.; Todd Hedge, P., Designing neural networks, Neural networks, 4, 53-60, (1991)
[19] D. Whitley, T. Starkweather, C. Bogart, Genetic algorithm and neural networks: optimizing connections and connectivity, Computing 14 (1989) 347-361.
[20] J.D. Schaffer, R.A. Caruana, L.J. Eshelman, Using genetic search to exploit the emergent behavior of neural networks, Physica D 42 (1990) 244-248.
[21] S. Harp, T. Samad, A. Guha, Toward the genetic synthesis of neural networks. In: Proceedings of the Third International Conference on Genetic Algorithms, J.D. Schaffer (Ed.), San Mateo, CA, Morgan Kaufman, 1989, pp. 762-767.
[22] H. Kitano, Designing neural networks using genetic algorithms with graph generation system, Complex Systems 4 (1990) 461-476. · Zbl 0709.92539
[23] H. Kitano, Evolution, complexity, entropy and artificial reality, Physica D 75 (1994) 239-263. · Zbl 0858.92024
[24] Mitchell, M., An introduction to genetic algorithms, (1995), MIT Press Cambridge, MA
[25] Arifovic, J., Genetic algorithm and the cobweb model, J. econ. dyn. control, 18, 3-28, (1994) · Zbl 0782.90017
[26] A.R. Gallant, H. White, There exists a neural network that does not make avoidable mistakes, Proceedings of the Second Annual IEEE Conference on Neural Networks, San Diego, CA, IEEE Press, New York, 1998, pp. I.657-I.664.
[27] Gallant, A.R.; White, H., On learning the derivatives of an unknown mapping with multilayer feedforward networks, Neural networks, 5, 129-138, (1992)
[28] Cybenko, G., Approximation by superposition of a sigmoidal function, Math. control, signals systems, 2, 303-314, (1989) · Zbl 0679.94019
[29] Funahashi, K.-I., On the approximate realization of continuous mappings by neural networks, Neural networks, 2, 183-192, (1989)
[30] Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural networks, 2, 359-366, (1989) · Zbl 1383.92015
[31] Hornik, K.; Stinchcombe, M.; White, H., Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural networks, 3, 551-560, (1990)
[32] Kuan, C.-M.; White, H., Artificial neural networks: an econometric perspective, Econometric rev., 13, 1-91, (1994) · Zbl 0832.62101
[33] White, H., Artificial neural networks: approximation & learning, (1992), Blackwell Cambridge
[34] Heerema, M.; van Leeuven, W.A., Derivation of Hebb’s rule, J. phys. A, 32, 263-286, (1999) · Zbl 0935.92002
[35] D.O. Hebb, The Organization of Behavior, New York, Wiley, 1949.
[36] White, H., Some asymptotic results for learning in single hidden layer feedforward network models, J. amer. statist. assoc., 94, 1003-1013, (1989) · Zbl 0721.62081
[37] Hénon, M., A two-dimensional mapping with a strange attactor, Commun. math. phys., 50, 69-77, (1976) · Zbl 0576.58018
[38] Oseledec, V.I., A multiplicative ergodic theorem. Liapunov characteristic numbers for dynamical system, Trans. Moscow math. soc., 19, 197-221, (1968)
[39] Raghunathan, M.S., A proof of Oseledec’s multiplicative ergodic theorem, Israel J. math., 32, 356-362, (1979) · Zbl 0415.28013
[40] Ruelle, D., Ergodic theory of differentiable dynamical systems, Publ. math. inst. hautes etudes scientifiques, 50, 27-58, (1979) · Zbl 0426.58014
[41] J.E. Cohen, J. Kesten, C.M. Newman (Eds.), Random Matrices and Their Application. Contemporary Mathematics, Vol. 50, American Mathematical Society, Providence, RI, 1986.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.