×

Obey validity limits of data-driven models through topological data analysis and one-class classification. (English) Zbl 1494.90088

Summary: Data-driven models are becoming increasingly popular in engineering, on their own or in combination with mechanistic models. Commonly, the trained models are subsequently used in model-based optimization of design and/or operation of processes. Thus, it is critical to ensure that data-driven models are not evaluated outside their validity domain during process optimization. We propose a method to learn this validity domain and encode it as constraints in process optimization. We first perform a topological data analysis using persistent homology identifying potential holes or separated clusters in the training data. In case clusters or holes are identified, we train a one-class classifier, i.e., a one-class support vector machine, on the training data domain and encode it as constraints in the subsequent process optimization. Otherwise, we construct the convex hull of the data and encode it as constraints. We finally perform deterministic global process optimization with the data-driven models subject to their respective validity constraints. To ensure computational tractability, we develop a reduced-space formulation for trained one-class support vector machines and show that our formulation outperforms common full-space formulations by a factor of over 3000, making it a viable tool for engineering applications. The method is ready-to-use and available open-source as part of our MeLOn toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

MSC:

90C26 Nonconvex programming, global optimization
90C90 Applications of mathematical programming
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Asprion, N., Modeling, simulation, and optimization 4.0 for a distillation column, Chem Ing Tech, 92, 7, 879-889 (2020)
[2] Asprion, N.; Böttcher, R.; Pack, R.; Stavrou, ME; Höller, J.; Schwientek, J.; Bortz, M., Gray-box modeling for the optimization of chemical processes, Chem Ing Tech, 91, 3, 305-313 (2019)
[3] Bhosekar A, Ierapetritou M (2018) Advances in surrogate based modeling, feasibility analysis, and optimization: a review. Comput Chem Eng 108:250-267
[4] Binchi, J.; Merelli, E.; Rucco, M.; Petri, G.; Vaccarino, F., jholes: a tool for understanding biological complex networks via clique weight rank persistent homology, Electron Notes Theor Comput Sci, 306, 5-18 (2014) · Zbl 1337.92073
[5] Bongartz D (2020) Deterministic global flowsheet optimization for the design of energy conversion processes. Ph.D. thesis, RWTH Aachen University
[6] Bongartz, D.; Mitsos, A., Deterministic global optimization of process flowsheets in a reduced space using McCormick relaxations, J Global Optim, 20, 9, 419 (2017) · Zbl 1386.90112
[7] Bongartz D, Najman J, Sass S, Mitsos A (2018) MAiNGO: McCormick-based algorithm for mixed integer nonlinear global optimization. Technical report, Process Systems Engineering (AVTSVT), RWTH Aachen University. http://permalink.avt.rwth-aachen.de/?id=729717
[8] Boukouvala, F.; Ierapetritou, MG, Feasibility analysis of black-box processes using an adaptive sampling kriging-based method, Comput Chem Eng, 36, 358-368 (2012)
[9] Cavanna NJ, Jahanseir M, Sheehy DR (2015) A geometric perspective on sparse filtrations. arXiv:1506.03797 · Zbl 1378.68163
[10] Chachuat, B.; Singer, AB; Barton, PI, Global methods for dynamic optimization and mixed-integer dynamic optimization, Ind Eng Chem Res, 45, 25, 8373-8392 (2006)
[11] Chachuat, B.; Houska, B.; Paulen, R.; Peric, N.; Rajyaguru, J.; Villanueva, ME, Set-theoretic approaches in analysis, estimation and control of nonlinear systems, IFAC-PapersOnLine, 48, 8, 981-995 (2015) · doi:10.1016/j.ifacol.2015.09.097
[12] Chambers EW, Letscher D (2018) Persistent homology over directed acyclic graphs. In: Research in computational topology. Springer, pp 11-32 · Zbl 1419.55009
[13] Chandola, V.; Banerjee, A.; Kumar, V., Anomaly detection: a survey, ACM Comput Sur (CSUR), 41, 3, 1-58 (2009)
[14] Charnes, A.; Cooper, WW, Chance-constrained programming, Manage Sci, 6, 1, 73-79 (1959) · Zbl 0995.90600
[15] Chazal F, Michel B (2017) An introduction to topological data analysis: fundamental and practical aspects for data scientists. arXiv:1710.04019
[16] Chen, Q.; Paulavičius, R.; Adjiman, CS; García-Muñoz, S., An optimization framework to combine operable space maximization with design of experiments, AIChE J, 64, 11, 3944-3957 (2018)
[17] Chollet F et al (2015) Keras. https://keras.io. Accessed May 2020
[18] Chung, MK; Hanson, JL; Ye, J.; Davidson, RJ; Pollak, SD, Persistent homology in sparse regression and its application to brain morphometry, IEEE Trans Med Imaging, 34, 9, 1928-1939 (2015)
[19] Cortes, C.; Vapnik, V., Support-vector networks, Mach Learn, 20, 3, 273-297 (1995) · Zbl 0831.68098
[20] Courrieu, P., Three algorithms for estimating the domain of validity of feedforward neural networks, Neural Netw, 7, 1, 169-174 (1994)
[21] Ding, X.; Li, Y.; Belatreche, A.; Maguire, LP, An experimental evaluation of novelty detection methods, Neurocomputing, 135, 313-327 (2014)
[22] Doncevic DT, Schweidtmann AM, Vaupel Y, Schäfer P, Caspari A, Mitsos A (2020) Deterministic global nonlinear model predictive control with recurrent neural networks embedded. In: IFAC conference proceedings (in press)
[23] Dreiseitl S, Osl M, Scheibböck C, Binder M (2010) Outlier detection with one-class svms: an application to melanoma prognosis. In: AMIA annual symposium proceedings, vol 2010. American Medical Informatics Association, p 172
[24] Epperly, TGW; Pistikopoulos, EN, A reduced space branch and bound algorithm for global optimization, J Global Optim, 11, 3, 287-311 (1997) · Zbl 1040.90567
[25] Evangelista PF, Embrechts MJ, Szymanski BK (2007) Some properties of the Gaussian kernel for one class learning. In: International conference on artificial neural networks. Springer, pp 269-278
[26] Fortuna, L.; Rizzo, A.; Sinatra, M.; Xibilia, M., Soft analyzers for a sulfur recovery unit, Control Eng Pract, 11, 12, 1491-1500 (2003)
[27] Fortuna L, Graziani S, Rizzo A, Xibilia MG (2007) Soft sensors for monitoring and control of industrial processes. Springer · Zbl 1136.93001
[28] Glassey J, Von Stosch M (2018) Hybrid modeling in process industries. CRC Press
[29] Hart WE, Laird CD, Watson JP, Woodruff DL, Hackebeil GA, Nicholson BL, Siirola JD (2017) Pyomo-optimization modeling in python, vol 67. Springer · Zbl 1370.90003
[30] Hiraoka, Y.; Nakamura, T.; Hirata, A.; Escolar, EG; Matsue, K.; Nishiura, Y., Hierarchical structures of amorphous solids characterized by persistent homology, Proc Natl Acad Sci, 113, 26, 7035-7040 (2016)
[31] Hüllen, G.; Zhai, J.; Kim, SH; Sinha, A.; Realff, MJ; Boukouvala, F., Managing uncertainty in data-driven simulation-based optimization, Comput Chem Eng (2019) · doi:10.1016/j.compchemeng.2019.106519
[32] Jin W, Barzilay R, Jaakkola T (2018) Junction tree variational autoencoder for molecular graph generation. arXiv:1802.04364
[33] Kahrs, O.; Marquardt, W., The validity domain of hybrid models and its application in process optimization, Chem Eng Process, 46, 11, 1054-1066 (2007)
[34] Kahrs, O.; Marquardt, W., Incremental identification of hybrid process models, Comput Chem Eng, 32, 4-5, 694-705 (2008)
[35] Kappatou CD, Bongartz D, Najman J, Sass S, Mitsos A (2020) Global dynamic optimization with hammerstein-wiener models embedded. http://www.optimization-online.org/DB_HTML/2020/09/8018.html
[36] Khan SS, Madden MG (2009) A survey of recent trends in one class classification. In: Irish conference on artificial intelligence and cognitive science. Springer, pp 188-197
[37] Khan, SS; Madden, MG, One-class classification: taxonomy of study and review of techniques, Knowl Eng Rev, 29, 3, 345-374 (2014)
[38] Kimura, Y.; Imai, K., Quantification of LSS using the persistent homology in the SDSS fields, Adv Space Res, 60, 3, 722-736 (2017)
[39] Knudde N, Couckuyt I, Shintani K, Dhaene T (2019) Active learning for feasible region discovery. In: 2019 18th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 567-572
[40] Kumar, JN; Li, Q.; Tang, KY; Buonassisi, T.; Gonzalez-Oyarce, AL; Ye, J., Machine learning enables polymer cloud-point engineering via inverse design, NPJ Comput Mater, 5, 1, 1-6 (2019)
[41] Larson BJ, Mattson CA (2012) Design space exploration for quantifying a system model’s feasible domain. ASME J Mech Des 134(4):041010. doi:10.1115/1.4005861
[42] Leonard, J.; Kramer, MA; Ungar, L., A neural network architecture that computes its own reliability, Comput Chem Eng, 16, 9, 819-835 (1992)
[43] Letscher, H.; Edelsbrunner, D.; Zomorodian, A., Topological persistence and simplification, Discrete Comput Geom, 28, 511-533 (2002) · Zbl 1011.68152
[44] Malak RJ Jr, Paredis CJJ (2010) Using support vector machines to formalize the valid input domain of predictive models in systems design problems. ASME J Mech Des 132(10):101001. doi:10.1115/1.4002151
[45] McBride, K.; Sundmacher, K., Overview of surrogate modeling in chemical process engineering, Chem Ing Tech, 91, 3, 228-239 (2019) · doi:10.1002/cite.201800091
[46] Mistry M, Letsios D, Krennrich G, Lee RM, Misener R (2018) Mixed-integer convex nonlinear optimization with gradient-boosted trees embedded. arXiv:1803.00952
[47] Mitsos, A.; Chachuat, B.; Barton, PI, McCormick-based relaxations of algorithms, SIAM J Optim, 20, 2, 573-601 (2009) · Zbl 1192.65083 · doi:10.1137/080717341
[48] Mogk G, Mrziglod T, Schuppert A (2002) Application of hybrid models in chemical industry. In: Computer aided chemical engineering, vol 10. Elsevier, pp 931-936
[49] Otter, N.; Porter, MA; Tillmann, U.; Grindrod, P.; Harrington, HA, A roadmap for the computation of persistent homology, EPJ Data Sci, 6, 1, 17 (2017)
[50] Papadopoulos, G.; Edwards, PJ; Murray, AF, Confidence estimation methods for neural networks: a practical comparison, IEEE Trans Neural Netw, 12, 6, 1278-1287 (2001)
[51] Patania, A.; Vaccarino, F.; Petri, G., Topological analysis of data, EPJ Data Sci, 6, 1-6 (2017) · Zbl 1440.55004
[52] Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V., Scikit-learn: machine learning in python, J Mach Learn Res, 12, 2825-2830 (2011) · Zbl 1280.68189
[53] Pimentel, MA; Clifton, DA; Clifton, L.; Tarassenko, L., A review of novelty detection, Sig Process, 99, 215-249 (2014)
[54] Pinto, J.; de Azevedo, CR; Oliveira, R.; von Stosch, M., A bootstrap-aggregated hybrid semi-parametric modeling framework for bioprocess development, Bioprocess Biosyst Eng, 42, 11, 1853-1865 (2019)
[55] Quaglio, M.; Fraga, ES; Cao, E.; Gavriilidis, A.; Galvanin, F., A model-based data mining approach for determining the domain of validity of approximated models, Chemometr Intell Lab Syst, 172, 58-67 (2018)
[56] Quek, C.; Balasubramanian, R.; Rangaiah, G., Consider using soft analyzers to improve SRU control, Hydrocarbon processing, 79, 1, 101-106 (2000)
[57] Rall, D.; Menne, D.; Schweidtmann, AM; Kamp, J.; von Kolzenberg, L.; Mitsos, A.; Wessling, M., Rational design of ion separation membranes, J Membr Sci, 569, 209-219 (2019)
[58] Roach, E.; Parker, RR; Malak, RJ Jr, An improved support vector domain description method for modeling valid search domains in engineering design problems, Int Des Eng Tech Conf Comput Inf Eng Conf, 54822, 741-751 (2011)
[59] Saadatfar, M.; Takeuchi, H.; Robins, V.; Francois, N.; Hiraoka, Y., Pore configuration landscape of granular crystallization, Nat Commun, 8, 1, 1-11 (2017)
[60] Schölkopf B (2001) The kernel trick for distances. In: Advances in neural information processing systems, pp 301-307
[61] Schölkopf B, Williamson RC, Smola AJ, Shawe-Taylor J, Platt JC (2000) Support vector method for novelty detection. In: Advances in neural information processing systems, pp 582-588
[62] Schweidtmann AM, Bongartz D, Grothe D, Kerkenhoff T, Lin X, Najman J, Mitsos A (2020a) Global optimization of Gaussian processes. arXiv:2005.10902 · Zbl 1476.90270
[63] Schweidtmann AM, Netze L, Mitsos A (2020b) Melon: Machine learning models for optimization. https://git.rwth-aachen.de/avt.svt/public/MeLOn/
[64] Schweidtmann AM, Rittig JG, König A, Grohe M, Mitsos A, Dahmen M (2020c) Graph neural networks for prediction of fuel ignition quality. ChemRxiv preprint ChemRxiv:12280325
[65] Schweidtmann, AM; Mitsos, A., Deterministic global optimization with artificial neural networks embedded, J Optim Theory Appl, 180, 3, 925-948 (2019) · Zbl 1407.90263
[66] Shahriari, B.; Swersky, K.; Wang, Z.; Adams, RP; de Freitas, N., Taking the human out of the loop: a review of bayesian optimization, Proc IEEE, 104, 1, 148-175 (2016) · doi:10.1109/JPROC.2015.2494218
[67] Simutis, R.; Havlik, I.; Schneider, F.; Dors, M.; Lübbert, A., Artificial neural networks of improved reliability for industrial process supervision, IFAC Proc Vol, 28, 3, 59-65 (1995)
[68] Smith AD, Dlotko P, Zavala VM (2020) Topological data analysis: concepts, computation, and applications in chemical engineering. arXiv:2006.03173
[69] Smola, AJ; Schölkopf, B., A tutorial on support vector regression, Stat Comput, 14, 3, 199-222 (2004)
[70] Tawarmalani, M.; Sahinidis, NV, A polyhedral branch-and-cut approach to global optimization, Math Program, 103, 2, 225-249 (2005) · Zbl 1099.90047 · doi:10.1007/s10107-005-0581-8
[71] Tax DMJ (2001) One-class classification: Concept learning in the absence of counter-examples. Ph.D. thesis, Delft University of Technology
[72] Tax, DM; Duin, RP, Data domain description using support vectors, ESANN, 99, 251-256 (1999)
[73] Teixeira, AP; Clemente, JJ; Cunha, AE; Carrondo, MJ; Oliveira, R., Bioprocess iterative batch-to-batch optimization based on hybrid parametric/nonparametric models, Biotechnol Prog, 22, 1, 247-258 (2006)
[74] Tralie, C.; Saul, N.; Bar-On, R., Ripser.py: a lean persistent homology library for python, J Open Source Softw, 3, 29, 925 (2018)
[75] Venkatasubramanian, V., The promise of artificial intelligence in chemical engineering: is it here, finally, AIChE J, 65, 2, 466-78 (2019)
[76] Virtanen, P.; Gommers, R.; Oliphant, TE; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J., Scipy 1.0: fundamental algorithms for scientific computing in python, Nat Methods, 17, 3, 261-272 (2020)
[77] Von Stosch, M.; Oliveira, R.; Peres, J.; de Azevedo, SF, Hybrid semi-parametric modeling in process systems engineering: past, present and future, Comput Chem Eng, 60, 86-101 (2014) · doi:10.1016/j.compchemeng.2013.08.008
[78] von Stosch, M.; Schenkendorf, R.; Geldhof, G.; Varsakelis, C.; Mariti, M.; Dessoy, S.; Vandercammen, A.; Pysik, A.; Sanders, M., Working within the design space: do our static process characterization methods suffice?, Pharmaceutics, 12, 6, 562 (2020)
[79] Wasserman, L., Topological data analysis, Ann Rev Stat Appl, 5, 501-532 (2018)
[80] Wilhelm ME, Stuber MD (2020) EAGO.jl: easy advanced global optimization in Julia. Optim Methods Softw. doi:10.1080/10556788.2020.1786566 · Zbl 1501.90077
[81] Xia, K., Persistent homology analysis of ion aggregations and hydrogen-bonding networks, Phys Chem Chem Phys, 20, 19, 13448-13460 (2018)
[82] Xia, K.; Anand, DV; Shikhar, S.; Mu, Y., Persistent homology analysis of osmolyte molecular aggregation and their hydrogen-bonding networks, Phys Chem Chem Phys, 21, 37, 21038-21048 (2019)
[83] Xiao, Y.; Wang, H.; Xu, W., Parameter selection of Gaussian kernel for one-class svm, IEEE Trans Cybern, 45, 5, 941-953 (2014)
[84] Xiao, Y.; Wang, H.; Zhang, L.; Xu, W., Two methods of selecting Gaussian kernel parameters for one-class svm and their application to fault detection, Knowl-Based Syst, 59, 75-84 (2014)
[85] Zhang, Q.; Grossmann, IE; Sundaramoorthy, A.; Pinto, JM, Data-driven construction of convex region surrogate models, Optim Eng, 17, 2, 289-332 (2016) · Zbl 1364.90231 · doi:10.1007/s11081-015-9288-8
[86] Zomorodian, A.; Carlsson, G., Computing persistent homology, Discrete Comput Geom, 33, 2, 249-274 (2005) · Zbl 1069.55003
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.