×

zbMATH — the first resource for mathematics

Rule extraction from recurrent neural networks: A taxonomy and review. (English) Zbl 1087.68089
Summary: Rule Extraction (RE) from Recurrent Neural Networks (RNNs) refers to finding models of the underlying RNN, typically in the form of finite state machines, that mimic the network to a satisfactory degree while having the advantage of being more transparent. RE from RNNs can be argued to allow a deeper and more profound form of analysis of RNNs than other, more or less ad hoc methods. RE may give us understanding of RNNs in the intermediate levels between quite abstract theoretical knowledge of RNNs as a class of computing devices and quantitative performance evaluations of RNN instantiations. The development of techniques for extraction of rules from RNNs has been an active field since the early 1990s. This article reviews the progress of this development and analyzes it in detail. In order to structure the survey and evaluate the techniques, a taxonomy specifically designed for this purpose has been developed. Moreover, important open research issues are identified that, if addressed properly, possibly can give the field a significant push forward.

MSC:
68T05 Learning and adaptive systems in artificial intelligence
Software:
LSTM
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] DOI: 10.1016/0950-7051(96)81920-4 · Zbl 05477666
[2] DOI: 10.1162/089976603321780281 · Zbl 1086.68580
[3] DOI: 10.1109/72.279181
[4] DOI: 10.1162/neco.1997.9.5.1127
[5] DOI: 10.1002/(SICI)1098-111X(200007)15:7<595::AID-INT2>3.0.CO;2-5 · Zbl 0958.68141
[6] DOI: 10.1162/neco.1995.7.4.845
[7] DOI: 10.1109/69.917555 · Zbl 05108850
[8] DOI: 10.1162/089976600300015097
[9] DOI: 10.1162/neco.1996.8.6.1135 · Zbl 05475401
[10] DOI: 10.1162/153244304773936063 · Zbl 1102.68487
[11] DOI: 10.1162/neco.1989.1.3.372
[12] DOI: 10.1016/0167-2789(94)90273-9 · Zbl 0860.68046
[13] Das S., Computer Science and Informatics 21 (2) pp 35– (1991)
[14] DOI: 10.1016/S0893-6080(97)00113-5
[15] DOI: 10.1016/0364-0213(90)90002-E
[16] DOI: 10.1007/BF00116897
[17] DOI: 10.1109/72.963769
[18] DOI: 10.1016/0893-6080(95)00041-0 · Zbl 05479128
[19] DOI: 10.1023/A:1010884214864 · Zbl 0983.68163
[20] DOI: 10.1162/neco.1992.4.3.393
[21] DOI: 10.1080/09540099308915703
[22] DOI: 10.1109/72.317740
[23] DOI: 10.1109/72.668898
[24] DOI: 10.1016/0893-6080(95)00025-U · Zbl 05479110
[25] DOI: 10.1109/72.286928
[26] DOI: 10.1162/08997660360675080 · Zbl 1085.68125
[27] DOI: 10.1016/0004-3702(90)90004-J · Zbl 05472899
[28] DOI: 10.1016/0921-8890(95)00016-9
[29] DOI: 10.1109/TNN.2003.813837
[30] Jagota A., Neural Computing Surveys 2 pp 1– (1999) · Zbl 0924.68156
[31] DOI: 10.1145/331499.331504
[32] DOI: 10.1080/09528139508953809
[33] DOI: 10.1162/089976601300014538 · Zbl 0973.68200
[34] DOI: 10.1109/69.842255 · Zbl 05108575
[35] DOI: 10.1162/neco.1994.6.6.1155
[36] DOI: 10.1007/BF02478259 · Zbl 0063.03860
[37] Medler D., Neural Computing Surveys 1 (1) pp 61– (1998)
[38] DOI: 10.1109/3477.499797
[39] DOI: 10.1142/S0218001493000431
[40] DOI: 10.1145/235809.235811 · Zbl 0883.68105
[41] DOI: 10.1016/0893-6080(95)00086-0 · Zbl 05477587
[42] DOI: 10.1109/69.485647 · Zbl 05108780
[43] DOI: 10.1109/91.660809
[44] DOI: 10.1016/S0019-9958(63)90290-0 · Zbl 0182.33602
[45] DOI: 10.1080/095400999116340
[46] DOI: 10.1162/neco.1992.4.2.234
[47] DOI: 10.1007/BF00114843
[48] DOI: 10.1006/jcss.1995.1013 · Zbl 0826.68104
[49] DOI: 10.1162/089976603322518731 · Zbl 1097.68589
[50] DOI: 10.1016/S0364-0213(99)00013-0 · Zbl 05396063
[51] DOI: 10.1109/72.728352
[52] DOI: 10.1109/TNN.2003.820839
[53] DOI: 10.1162/08997660360675099 · Zbl 1085.68136
[54] DOI: 10.1109/72.750555
[55] DOI: 10.1162/neco.1995.7.4.822
[56] Tio P., Neural Network World 8 (5) pp 517– (1998)
[57] Towell G. G., Machine Learning 13 (1) pp 17– (1993)
[58] DOI: 10.1162/08997660460733994 · Zbl 1084.68110
[59] DOI: 10.1016/0960-0779(94)90015-9 · Zbl 0793.60117
[60] DOI: 10.1162/neco.1993.5.6.976
[61] DOI: 10.1177/1059712302010003003
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.