×

zbMATH — the first resource for mathematics

A multiscale neural network based on hierarchical matrices. (English) Zbl 1435.65181
MSC:
65M75 Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs
60H35 Computational methods for stochastic equations (aspects of stochastic analysis)
35Q92 PDEs in connection with biology, chemistry and other natural sciences
92B20 Neural networks for/in biological studies, artificial life and related topics
35Q55 NLS equations (nonlinear Schrödinger equations)
82M36 Computational density functional analysis in statistical mechanics
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng, Tensorflow: A system for large-scale machine learning, in Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), USENIX Association, 2016, pp. 265-283.
[2] J. R. Anglin and W. Ketterle, Bose-Einstein condensation of atomic gases, Nature, 416 (2002), pp. 211-218.
[3] M. Araya-Polo, J. Jennings, A. Adler, and T. Dahlke, Deep-learning tomography, The Leading Edge, 37 (2018), pp. 58-66, https://doi.org/10.1190/tle37010058.1.
[4] V. Badrinarayanan, A. Kendall, and R. Cipolla, SegNet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), pp. 2481-2495.
[5] W. Bao and Q. Du, Computing the ground state solution of Bose-Einstein condensates by a normalized gradient flow, SIAM J. Sci. Comput., 25 (2004), pp. 1674-1697, https://doi.org/10.1137/S1064827503422956. · Zbl 1061.82025
[6] M. Barrault, Y. Maday, N. C. Nguyen, and A. T. Patera, An “empirical interpolation” method: Application to efficient reduced-basis discretization of partial differential equations, C. R. Math. Acad. Sci. Paris, 339 (2004), pp. 667-672, https://doi.org/10.1016/j.crma.2004.08.006. · Zbl 1061.65118
[7] C. Beck, W. E, and A. Jentzen, Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations, J. Nonlinear Sci., 29 (2017), pp. 1563-1619. · Zbl 1442.91116
[8] J. Berg and K. Nyström, A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing, 317 (2018), pp. 28-41.
[9] S. Börm, L. Grasedyck, and W. Hackbusch, Introduction to hierarchical matrices with applications, Eng. Anal. Bound. Elem., 27 (2003), pp. 405-422. · Zbl 1035.65042
[10] A. Brandt, Multi-level adaptive solutions to boundary-value problems, Math. Comp., 31 (1977), pp. 333-390. · Zbl 0373.65054
[11] J. Bruna and S. Mallat, Invariant scattering convolution networks, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013), pp. 1872-1886.
[12] P. Chaudhari, A. Oberman, S. Osher, S. Soatto, and G. Carlier, Partial differential equations for training deep neural networks, in Proceedings of the 2017 51st Asilomar Conference on Signals, Systems, and Computers, IEEE, 2017, pp. 1627-1631, https://doi.org/10.1109/ACSSC.2017.8335634. · Zbl 1427.82032
[13] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Trans. Pattern Anal. Mach. Intell., 40 (2018), pp. 834-848, https://doi.org/10.1109/TPAMI.2017.2699184.
[14] F. Chollet et al., Keras, 2015, https://keras.io.
[15] N. Cohen, O. Sharir, and A. Shashua, On the expressive power of deep learning: A tensor analysis, in Proceedings of the 29th Annual Conference on Learning Theory, 2016, pp. 698-728.
[16] T. Dozat, Incorporating Nesterov Momentum into Adam, International Conference on Learning Representations, in Proceedings of the 4th International Conference on Learning Representations, Workshop Track, 2016.
[17] W. E, J. Han, and A. Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Commun. Math. Stat., 5 (2017), pp. 349-380, https://doi.org/10.1007/s40304-017-0117-6. · Zbl 1382.65016
[18] Y. Efendiev, J. Galvis, G. Li, and M. Presho, Generalized multiscale finite element methods. Nonlinear elliptic equations, Commun. Comput. Phys., 15 (2014), pp. 733-755, https://doi.org/10.4208/cicp.020313.041013a. · Zbl 1388.65146
[19] Y. Efendiev and T. Hou, Multiscale finite element methods for porous media flows and their applications, Appl. Numer. Math., 57 (2007), pp. 577-596, https://doi.org/10.1016/j.apnum.2006.07.009. · Zbl 1112.76046
[20] Y. Fan, J. Feliu-Fabà, L. Lin, L. Ying, and L. Zepeda-Nún͂ez, A multiscale neural network based on hierarchical nested bases, Res. Math. Sci., 6 (2019), 21. · Zbl 07096701
[21] M. Fenn and G. Steidl, FMM and \(\mathcal H\)-Matrices: A Short Introduction to the Basic Idea, Tech. Report TR-2002-008, Department for Mathematics and Computer Science, University of Mannheim, Mannheim, Germany, 2002, https://ub-madoc.bib.uni-mannheim.de/744/1/TR-02-008.pdf.
[22] L. Greengard and V. Rokhlin, A fast algorithm for particle simulations, J. Comput. Phys., 73 (1987), pp. 325-348. · Zbl 0629.65005
[23] M. A. Grepl, Y. Maday, N. C. Nguyen, and A. T. Patera, Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations, M2AN Math. Model. Numer. Anal., 41 (2007), pp. 575-605, https://doi.org/10.1051/m2an:2007031. · Zbl 1142.65078
[24] W. Hackbusch, A sparse matrix arithmetic based on \(\mathcal H\)-matrices. Part I: Introduction to \(\mathcal H\)-matrices, Computing, 62 (1999), pp. 89-108. · Zbl 0927.65063
[25] W. Hackbusch, L. Grasedyck, and S. Börm, An introduction to hierarchical matrices, Math. Bohem., 127 (2002), pp. 229-241. · Zbl 1007.65032
[26] W. Hackbusch and B. N. Khoromskij, A sparse \(\mathcal H\)-matrix arithmetic: General complexity estimates, J. Comput. Appl. Math., 125 (2000), pp. 479-501. · Zbl 0977.65036
[27] J. Han, A. Jentzen, and W. E, Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. USA, 115 (2018), pp. 8505-8510. · Zbl 1416.35137
[28] K. He and J. Sun, Convolutional neural networks at constrained time cost, in Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015, pp. 5353-5360.
[29] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016, pp. 770-778.
[30] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016, pp. 770-778.
[31] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal Processing Mag., 29 (2012), pp. 82-97, https://doi.org/10.1109/MSP.2012.2205597.
[32] P. Hohenberg and W. Kohn, Inhomogeneous electron gas, Phys. Rev. (2), 136 (1964), pp. B864-B871.
[33] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks, 4 (1991), pp. 251-257, https://doi.org/10.1016/0893-6080(91)90009-T.
[34] Y. Khoo, J. Lu, and L. Ying, Solving Parametric PDE Problems with Artificial Neural Networks, preprint, https://arxiv.org/abs/1707.03351, 2017.
[35] V. Khrulkov, A. Novikov, and I. Oseledets, Expressive Power of Recurrent Neural Networks, preprint, https://arxiv.org/abs/1711.00811, 2017.
[36] N. Kishore Kumar and J. Schneider, Literature survey on low rank approximation of matrices, Linear Multilinear Algebra, 65 (2017), pp. 2212-2244. · Zbl 1387.65039
[37] W. Kohn and L. J. Sham, Self-consistent equations including exchange and correlation effects, Phys. Rev. (2), 140 (1965), pp. A1133-A1138.
[38] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, Curran Associates, Red Hook, NY, 2012, pp. 1097-1105.
[39] Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, 521 (2015), pp. 436-444.
[40] M. K. K. Leung, H. Y. Xiong, L. J. Lee, and B. J. Frey, Deep learning of the tissue-regulated splicing code, Bioinformatics, 30 (2014), pp. i121-i129, https://doi.org/10.1093/bioinformatics/btu277.
[41] Y. Li, X. Cheng, and J. Lu, Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks, preprint, https://arxiv.org/abs/1805.07451, 2018.
[42] L. Lin, J. Lu, and L. Ying, Fast construction of hierarchical matrix representation from matrix-vector multiplication, J. Comput. Phys., 230 (2011), pp. 4071-4087. · Zbl 1218.65038
[43] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez, A survey on deep learning in medical image analysis, Med. Image Anal., 42 (2017), pp. 60-88, https://doi.org/10.1016/j.media.2017.07.005.
[44] J. Ma, R. P. Sheridan, A. Liaw, G. E. Dahl, and V. Svetnik, Deep neural nets as a method for quantitative structure-activity relationships, J. Chem. Inf. Model., 55 (2015), pp. 263-274, https://doi.org/10.1021/ci500747n.
[45] Y. Maday, O. Mula, and G. Turinici, Convergence analysis of the generalized empirical interpolation method, SIAM J. Numer. Anal., 54 (2016), pp. 1713-1731, https://doi.org/10.1137/140978843. · Zbl 1347.41044
[46] S. Mallat, A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed., Academic Press, Boston, 2009, https://doi.org/10.1016/B978-0-12-374370-1.50001-9. · Zbl 1170.94003
[47] M. Mathieu, C. Couprie, and Y. LeCun, Deep Multi-Scale Video Prediction Beyond Mean Square Error, preprint, http://arxiv.org/abs/1712.04741, 2016.
[48] H. Mhaskar, Q. Liao, and T. Poggio, Learning Functions: When Is Deep Better Than Shallow, preprint, https://arxiv.org/abs/1603.00988, 2016.
[49] P. Paschalis, N. D. Giokaris, A. Karabarbounis, G. Loudos, D. Maintas, C. Papanicolas, V. Spanoudaki, C. Tsoumpas, and E. Stiliaris, Tomographic image reconstruction using artificial neural networks, in Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 527 (2004), pp. 211-215, https://doi.org/10.1016/j.nima.2004.03.122.
[50] D. M. Pelt and J. A. Sethian, A mixed-scale dense convolutional neural network for image analysis, Proc. Natl. Acad. Sci. USA, 115 (2018), pp. 254-259, https://doi.org/10.1073/pnas.1715832114.
[51] L. Pitaevskii, Vortex lines in an imperfect Bose gas, J. Exp. Theor. Phys., 13 (1961), pp. 451-454.
[52] M. Raissi, Forward-Backward Stochastic Neural Networks: Deep Learning of High-Dimensional Partial Differential Equations, preprint, https://arxiv.org/abs/1804.07010, 2018.
[53] M. Raissi and G. E. Karniadakis, Hidden physics models: Machine learning of nonlinear partial differential equations, J. Comput. Phys., 357 (2018), pp. 125-141, https://doi.org/10.1016/j.jcp.2017.11.039. · Zbl 1381.68248
[54] O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds., Springer, Cham, 2015, pp. 234-241.
[55] K. Rudd, G. D. Muro, and S. Ferrari, A constrained backpropagation approach for the adaptive solution of partial differential equations, IEEE Trans. Neural Netw. Learn. Syst., 25 (2014), pp. 571-584, https://doi.org/10.1109/TNNLS.2013.2277601.
[56] J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, 61 (2015), pp. 85-117, https://doi.org/10.1016/j.neunet.2014.09.003.
[57] J. Sirignano and K. Spiliopoulos, DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375 (2018), pp. 1339-1364. · Zbl 1416.65394
[58] I. Sutskever, O. Vinyals, and Q. V. Le, Sequence to sequence learning with neural networks, in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, eds., Curran Associates, Red Hook, NY, 2014, pp. 3104-3112.
[59] L. N. Trefethen, Spectral Methods in MATLAB, SIAM, Philadelphia, 2000, https://doi.org/10.1137/1.9780898719598. · Zbl 0953.68643
[60] D. Ulyanov, A. Vedaldi, and V. Lempitsky, Deep image prior, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 9446-9454.
[61] Y. Wang, C. W. Siu, E. T. Chung, Y. Efendiev, and M. Wang, Deep Multiscale Model Learning, preprint, https://arxiv.org/abs/1806.04830, 2018.
[62] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, Understanding Deep Learning Requires Rethinking Generalization, preprint, https://arxiv.org/abs/1611.03530, 2016.
[63] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising, IEEE Trans. Image Process., 26 (2017), pp. 3142-3155. · Zbl 1409.94754
[64] L. Zhang, J. Han, H. Wang, R. Car, and W. E, DeePCG: Constructing coarse-grained models via deep neural networks, J. Chem. Phys., 149 (2018), 034101.
[65] L. Zhang, H. Wang, and W. E, Adaptive coupling of a deep neural network potential to a classical force field, J. Chem. Phys., 149 (2018) 154107.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.