×

Review of research in the field of developing methods to extract rules from artificial neural networks. (English. Russian original) Zbl 07516576

J. Comput. Syst. Sci. Int. 60, No. 6, 966-980 (2021); translation from Izv. Ross. Akad. Nauk, Teor. Sist. Upr. 2021, No. 6, 106-121 (2021).

MSC:

68Txx Artificial intelligence

Software:

ANFIS; HyFIS
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] M. Bilgic and R. J. Mooney, “Explaining recommendations: Satisfaction vs. promotion,” in Proceedings of the Beyond Personalization Workshop,2005, Vol. 5, p. 153.
[2] Swartout, W. R.; Moore, J. D., Second Generation Expert Systems (1993), Berlin: Springer, Berlin · Zbl 0819.68132 · doi:10.1007/978-3-642-77927-5_24
[3] Chandrasekaran, B.; Tanner, M. C.; Josephson, J. R., Explaining control strategies in problem solving, IEEE Expert, 4, 9-15 (1989) · doi:10.1109/64.21896
[4] Dhaliwal, J. S.; Benbasat, I., The use and effects of knowledge-based system explanations: theoretical foundations and a framework for empirical evaluation, Inform. Syst. Res., 7, 342-362 (1996) · doi:10.1287/isre.7.3.342
[5] Eining, M. M.; Dorr, P. B., The impact of expert system usage on experiential learning in an auditing setting, Inform. Syst., 5, 1-16 (1991)
[6] Murphy, D. S., Expert system use and the development of expertise in auditing: A preliminary investigation, Inform. Syst., 4, 18-35 (1990)
[7] Lamberti, D. M.; Walace, W. A., Intelligent interface design: An empirical assessment of knowledge presentation in expert systems, MIS Quart., 14, 279-311 (1990) · doi:10.2307/248891
[8] Zakharov, V. N.; Popov, E. V.; Pospelov, D. A.; Khoroshevskii, V. F., Artificial Intelligence, The Reference Book (1990), Moscow: Radio Svyaz, Moscow
[9] Popov, E. V., Expert Systems: Solving Unformalized Tasks in a Dialogue with a Computer (1987), Moscow: Nauka, Moscow
[10] W. R. Swartout, “A digitalis therapy advisor with explanations,” in Proceedings of the 5th International Joint Conference on Artificial Intelligence (Cambridge, 1977), Vol. 2, pp. 819-825.
[11] Weiner, J. L., BLAH, A system that explains its reasoning, Artif. Intell., 15, 19-48 (1980) · doi:10.1016/0004-3702(80)90021-1
[12] Swartout, W. R.; Paris, C.; Moore, J., Explanations in knowledge systems: Design for explainable expert systems, IEEE Expert, 6, 58-64 (1991) · doi:10.1109/64.87686
[13] W. J. Clancey, Intelligent Tutoring Systems: A Tutorial Survey (Stanford Univ. Dep. Comput. Sci., Stanford, 1986).
[14] R. Sinha and K. Swearingen, “The role of transparency in recommender systems,” in Extended Abstracts of CHI’02 Conference on Human Factors in Computing Systems (Minneapolis, 2002), pp. 830-831.
[15] Gruber, T., Learning why by being told what, IEEE Expert, 6, 65-75 (1991) · doi:10.1109/64.85922
[16] Clancey, W. J., Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project (1984), Reading, MA: Addison-Wesley, Reading, MA
[17] Clancey, W. J., From GUIDON to NEOMYCIN and HERACLES in twenty short lessons, AI Magazine, 7, 40 (1986)
[18] Arioua, A.; Buche, P.; Croitoru, M., Explanatory dialogs with argumentative faculties over inconsistent knowledge bases, Expert Syst. Appl., 80, 244-262 (2017) · doi:10.1016/j.eswa.2017.03.009
[19] U. Johansson, T. Lofstrom, R. Konig, C. Sonstro, and L. Nilsson, “Rule extraction from opaque models-a slightly different perspective,” in Proceedings of the 5th International Conference on Machine Learning and Applications (ICMLA’06), Orlando, FL, USA,2006, pp. 22-27.
[20] M. Craven and J. Shavlik, “Rule extraction: Where do we go from here,” in University of Wisconsin Machine Learning Research Group Working Paper (Wisconsin, 1999), pp. 99-108.
[21] Andrew, R.; Diederich, J.; Tickle, A. B., Survey and critique of techniques for extracting rules from trained artificial networks, Knowledge-based Syst., 8, 373-389 (1995) · doi:10.1016/0950-7051(96)81920-4
[22] S. Thrum, “Extracting provably correct rules from artificial neural networks,” Technical Report (Inst. Inform. III, Bonn, 1993).
[23] M. Craven and J. W. Shavlik, “Using sampling and queries to extract rules from trained neural networks,” in Proceedings of the 11th International Conference, Rutgers Univ., New Brunswick, USA,1994, pp. 37-45.
[24] Fu, L., Rule generation from neural networks, IEEE Trans. Syst. Man Cybern., 24, 1114-1124 (1994) · doi:10.1109/21.299696
[25] M. Sato and H. Tsukimoto, “Rule extraction from neural networks via decision tree induction,” in Proceedings of International Joint Conference on Neural Networks (IJCNN’01) (Washington, DC, 2001), Vol. 3, pp. 1870-1875.
[26] Tickle, A. B.; Andrew, R.; Golea, M.; Diederich, J., The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks, IEEE Trans. Neural Networks, 9, 1057-1068 (1998) · doi:10.1109/72.728352
[27] K. K. Sethi, D. K. Mishra, and B. Mishra, “KDRuleEx: A novel approach for enhancing user comprehensibility using rule extraction,” in Intelligent Systems, Modelling and Simulation (ISMS), Proceedings of the 3rd International Conference, Kota Kinabalu, Malaysia,2012, pp. 55-60.
[28] U. Johansson, T. Lofstrom, R. Konig, C. Sonstrod, and L. Niklasson, “Rule extraction from opaque models-a slightly different perspective,” in Proceedings of the 5th International Conference on Machine Learning and Applications, ICMLA’06, Orlando, FL, USA,2006, pp. 22-27.
[29] M. Rangwala and G. R. Weckman, “Extracting rules from artificial neural networks utilizing TREPAN,” in Proceedings of IIE Annual Conference, Orlando, Florida,2006.
[30] Hailesilassie, T., Extraction algorithm for deep neural networks: A review, Int. J. Comput. Sci. Inform. Secur., 14, 376-381 (2016)
[31] Averkin, A.; Yarushev, S., Hybrid neural networks and time series forecasting, Commun. Comput. Inform. Sci., 934, 230-239 (2018) · doi:10.1007/978-3-030-00617-4_21
[32] G. Pilato, S. A. Yarushev, and A. N. Averkin, “Prediction and detection of user emotions based on neuro-fuzzy neural networks in social networks,” in Proceedings of the 3rd International Scientific Conference on Intelligent Information Technologies for Industry IITI’18, Sochi, Russia, Adv. Intell. Syst. Comput. 875, 118-126 (2018).
[33] A. N. Averkin, G. Pilato, and S. A. Yarushev, “An approach for prediction of user emotions based on ANFIS in social networks,” in Proceedings of the 2nd International Scientific and Practical Conference on Fuzzy Technologies in the Industry FTI 2018-CEUR Workshop, Ostrava-Prague, Czech Republic,2018, pp. 126-134.
[34] Jin, X.-H., Neurofuzzy decision support system for efficient risk allocation in public-private partnership infrastructure projects, J. Comput. Civ. Eng., 24, 525-538 (2010) · doi:10.1061/(ASCE)CP.1943-5487.0000058
[35] X.-H. Jin, “Model for efficient risk allocation in privately financed public infrastructure projects using neuro-fuzzy techniques,” J. Constr. Eng. Manag., 1003-1014 (2011).
[36] V. V. Borisov, A. S. Fedulov, and M. M. Zernov, Fundamentals of Hybridization of Fuzzy Models, Vol. 9 of Fundamentals of Fuzzy Mathematics Series (Goryachaya Liniya-Telekom, Moscow, 2017) [in Russian].
[37] Rutkowska, D.; Piliński, M.; Rutkowski, L., Neural Networks, Genetic Algorithms and Fuzzy Systems (2008), Warszawa: Naukowe PWN, Warszawa
[38] Rajab, S.; Sharma, V., A review on the applications of neuro-fuzzy systems in business, Artif. Intell. Rev., 49, 481-510 (2018) · doi:10.1007/s10462-016-9536-0
[39] Mitra, S.; Hayashi, Y., Neuro-fuzzy rule generation: Survey in soft computing framework, IEEE Trans. Neural Network, 11, 748-768 (2000) · doi:10.1109/72.846746
[40] Vieira, J.; Morgado-Dias, F.; Mota, A., Neuro-fuzzy systems: A survey, WSEAS Trans. Syst., 3, 414-419 (2004)
[41] Kim, J.; Kasabov, N., HyFIS: Adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems, Neural Network, 12, 1301-1319 (1999) · doi:10.1016/S0893-6080(99)00067-2
[42] Shihabudheen, K. V.; Pillai, G. N., “Recent advances in neuro-fuzzy system: A survey,” Knowl.-, Based Syst., 152, 136-162 (2018) · doi:10.1016/j.knosys.2018.04.014
[43] Batyrshin, I. Z.; Nedosekin, A. O.; Stetsko, A. A.; Tarasov, V. B.; Yazenin, A. V.; Yarushkina, N. G., Fuzzy Hybride Systems: Theory and Practice (2007), Moscow: Fizmatlit, Moscow · Zbl 1137.93001
[44] Viharos, Z. J.; Kis, K. B., Survey on neuro-fuzzy systems and their applications in technical diagnostics and measurement, Measurement, 67, 126-136 (2015) · doi:10.1016/j.measurement.2015.02.001
[45] Lin, C. T.; Lee, C. S. G., Neural network based fuzzy logic control and decision system, IEEE Trans. Comput., 40, 1320-1336 (1991) · Zbl 1395.93324 · doi:10.1109/12.106218
[46] Jang, J.-S. R., ANFIS: Adaptive-network-based fuzzy inference system, IEEE Trans. Syst. Cybern., 23, 665-685 (1993) · doi:10.1109/21.256541
[47] H. Naderpour and M. Mirrashid, “Shear failure capacity prediction of concrete beam-column joints in terms of ANFIS and GMDH,” Pract. Period. Struct. Des. Constr. 24 (2) (2019).
[48] L. Fan, “Revisit fuzzy neural network: Demystifying batch normalization and ReLU with generalized hamming network,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA,2017, pp. 1920-1929.
[49] Bherenji, H. R.; Khedkar, P., Learning and tuning fuzzy logic controllers through reinforcements, IEEE Trans. Neural Networks, 3, 724-740 (1992) · doi:10.1109/72.159061
[50] Nauck, D.; Kruse, R., Neuro-fuzzy systems for function approximation, Fuzzy Sets Syst., 101, 261-271 (1999) · Zbl 0937.93029 · doi:10.1016/S0165-0114(98)00169-9
[51] Tano, S.; Oyama, T.; Arnould, T., Deep combination of fuzzy inference and neural network in fuzzy inference, Fuzzy Sets Syst., 82, 151-160 (1996) · doi:10.1016/0165-0114(95)00251-0
[52] Feng, J. Ch.; Teng, L. Ch., An online self constructing neural fuzzy inference network and its applications, IEEE Trans. Fuzzy Syst., 6, 12-32 (1998) · doi:10.1109/91.660805
[53] N. Kasabov and Qun Song, “Dynamic evolving fuzzy neural networks with ”m-out-of-n’ activation nodes for on-line adaptive systems,” Technical Report TR99/04 (Department of Inform. Sci., Univ. Otago, Otago, 1999).
[54] Gunning, D.; Aha, D., DARPA’S explainable artificial intelligence (XAI) program, AI Magazine, 40, 44-58 (2019) · doi:10.1609/aimag.v40i2.2850
[55] A. N. Gorban’, “Errors of data-based intelligence,” in Proceedings of the International Conference on Intelligent Systems in Science and Technology, and the 6th All-Russian Scientific and Practical Conference on Artificial Intelligence in Solving Urgent Social and Economic Problems of the XXI Century, Perm,2020, pp. 11-13.
[56] R. Hu, J. Andreas, M. Rohrbach, T. Darrell, and K. Saenko, “Learning to reason: End-to-end module networks for visual question answering,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, New York, 2017), pp. 804-813.
[57] J. Kim and J. Canny, “Interpretable learning for self-driving cars by visualizing causal attention,” in Proceedings of the International Conference on Computer Vision (IEEE, New York, 2017), pp. 2942-2950.
[58] L. A. Hendricks, T. Darrell, and Z. Akata, “Grounding visual explanations,” in Proceedings of the European Conference of Computer Vision (ECCV), Munich, Germany (Springer, 2018).
[59] K. Marazopoulou, M. Maier, and D. Jensen, “Learning the structure of causal models with relational and temporal dependence,” in Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, Association for Uncertainty in Artificial Intelligence, Amsterdam, Netherlands,2015, pp. 572-581.
[60] Pfeffer, A., Practical Probabilistic Programming (2016), Greenwich, CT: Manning, Greenwich, CT
[61] M. Harradon, J. Druce, and B. Ruttenberg, “Causal learning and explanation of deep neural networks via autoencoded activations,” arXiv: 1802.00541v1 [cs.AI] (2018).
[62] L. She and J. Y. Chai, “Interactive learning for acquisition of grounded verb semantics towards human-robot communication,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017, Vol. 1, pp. 1634-1644.
[63] Z. Qi and F. Li, “Learning explainable embeddings for deep networks,” in Proceedings of the NIPS Workshop on Interpreting, Explaining and Visualizing Deep Learning, Long Beach,2017.
[64] J. Dodge, S. Penney, C. Hilderbrand, A. Anderson, and M. Burnett, “How the experts do it: Assessing and explaining agent behaviors in real-time strategy games,” in Proceedings of the CHI Conference on Human Factors in Computing Systems (Assoc. for Comput. Machinery, New York, 2018), pp. 1-12.
[65] F. Belbute-Peres and J. Z. Kolter, “A modular differentiable rigid body physics engine,” in Neural Information Processing Systems, Deep Reinforcement Learning Symposium, Long Beach, CA,2017.
[66] A. Hefny, Z. Marinho, W. Sun, S. Srinivasa, and G. Gordon, “Recurrent predictive state policy networks,” in Proceedings of the 35th International Conference on Machine Learning (Int. Machine Learning Soc., Stockholm, Sweden, 2018), pp. 1949-1958.
[67] P. Vicol, M. Tapaswi, L. Castrejon, and S. Fidler, “MovieGraphs: Towards understanding human-centric situations from videos,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, New York, 2018), pp. 4631-4640.
[68] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Object detectors emerge in deep scene CNNs,” in Proceedings of the International Conference on Learning Representations, San Diego, CA,2015.
[69] Gogate, V.; Domingos, P., Probabilistic theorem proving, Commun. ACM, 59, 107-15 (2016) · doi:10.1145/2936726
[70] M. Du, N. Liu, Q. Song, and X. Hu, “Towards Explanation of DNN-based prediction and guided feature inversion,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Assoc. Comput. Machinery, New York, 2018), pp. 1358-1367.
[71] Gao, J.; Liu, N.; Lawley, M.; Hu, X., An interpretable classification framework for information extraction from online healthcare forums, J. Healthcare Eng., 2017, 2460174 (2017)
[72] S. C.-H. Yang and P. Shafto, “Explainable artificial intelligence via Bayesian teaching,” in Proceedings of the 31st Conference on Neural Information Processing Systems Workshop on Teaching Machines, Robots and Humans, Long Beach, CA,2017.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.