×

Asymptotic theory of \(\ell_1\)-regularized PDE identification from a single noisy trajectory. (English) Zbl 1493.62440

Summary: We provide a formal theoretical analysis on the PDE identification via the \(\ell_1\)-regularized pseudo least square method from the statistical point of view. In this article, we assume that the differential equation governing the dynamic system can be represented as a linear combination of various linear and nonlinear differential terms. Under noisy observations, we employ local-polynomial fitting for estimating state variables and apply the \(\ell_1\) penalty for model selection. Our theory proves that the classical mutual incoherence condition on the feature matrix \(\boldsymbol{F}\) and the \(\boldsymbol{\beta^*}_{\min}\)-condition for the ground-truth signal \(\boldsymbol{\beta^*}\) are sufficient for the signed-support recovery of the \(\ell_1\)-PsLS method. We run numerical experiments on two popular PDE models, the viscous Burgers and the Korteweg-de Vries (KdV) equations, and the results from the experiments corroborate our theoretical predictions.

MSC:

62J07 Ridge regression; shrinkage estimators (Lasso)
93B30 System identification
35G35 Systems of linear higher-order PDEs

Software:

PDCO
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Schrödinger, E., An undulatory theory of the mechanics of atoms and molecules, Phys. Rev., 28 (1926), pp. 1049. · JFM 52.0965.07
[2] Black, F. and Scholes, M., The pricing of options and corporate liabilities, J. Polit. Econ., 81 (1973), pp. 637-654. · Zbl 1092.91524
[3] Haskovec, J., Kreusser, L. M., and Markowich, P., ODE and PDE Based Modeling of Biological Transportation Networks, preprint, https://arxiv.org/abs/1805.08526, 2018. · Zbl 1433.35421
[4] Achdou, Y., Buera, F. J., Lasry, J.-M., Lions, P.-L., and Moll, B., Partial differential equation models in macroeconomics, Philos. Trans. Royal Soc. A Math. Phys. Eng. Sci., 372 (2014), 20130397. · Zbl 1353.91027
[5] Musha, T. and Higuchi, H., Traffic current fluctuation and the Burgers equation, Japan. J. Appl. Phys., 17 (1978), 811.
[6] Tikhomirov, V. M., A study of the diffusion equation with increase in the amount of substance, and its application to a biological problem, in Selected Works of AN Kolmogorov Volume II, Springer, New York, 1991, pp. 242-270.
[7] Newell, A. C., Solitons in Mathematics and Physics, , SIAM, Philadelphia, 1985. · Zbl 0565.35003
[8] Fan, J., Gasser, T., Gijbels, I., Brockmann, M., and Engel, J., Local polynomial regression: Optimal kernels and asymptotic minimax efficiency, Ann. Inst. Stat. Math., 49 (1997), pp. 79-99. · Zbl 0890.62032
[9] Fan, J., Local Polynomial Modelling and Its Applications, Monogr. Stat. Appl. Probab. 66, Chapman & Hall, London, 2018.
[10] Liang, H. and Wu, H., Parameter estimation for differential equation models using a framework of measurement error in regression models, J. Amer. Stat. Assoc., 103 (2008), pp. 1570-1583. · Zbl 1286.62039
[11] Chen, J. and Wu, H., Efficient local estimation for time-varying coefficients in deterministic dynamic models with applications to HIV-1 dynamics, J. Amer. Stat. Assoc., 103 (2008), pp. 369-384. · Zbl 1469.62365
[12] Chen, J. and Wu, H., Estimation of time-varying parameters in deterministic dynamic models, Stat. Sinica, 18 (2008), pp. 987-1006. · Zbl 1149.62079
[13] Bär, M., Hegger, R. and Kantz, H., Fitting partial differential equations to space-time dynamics, Phys. Rev. E, 59 (1999), pp. 337.
[14] Tibshirani, R., Regression shrinkage and selection via the Lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 58 (1996), pp. 267-288. · Zbl 0850.62538
[15] Wainwright, M. J., Sharp thresholds for high-dimensional and noisy sparsity recovery using \(\ell_1\) -constrained quadratic programming (Lasso), IEEE Trans. Inform. Theory, 55 (2009), pp. 2183-2202. · Zbl 1367.62220
[16] Jia, J., Rohe, K., and Yu, B., The lasso under Poisson-like heteroscedasticity, Stat. Sinica, 23 (2013), pp. 99-118. · Zbl 1259.62042
[17] Bühlmann, P. and Van De Geer, S., Statistics for High-Dimensional Data: Methods: Theory and Applications, Springer, Heidelberg, 2011. · Zbl 1273.62015
[18] Ravikumar, P., Wainwright, M. J., and Lafferty, J. D., High-dimensional Ising model selection using \(\ell_1\) -regularized logistic regression, Ann. Stat., 38 (2010), pp. 1287-1319. · Zbl 1189.62115
[19] Ravikumar, P., Lafferty, J., Liu, H., and Wasserman, L., Sparse additive models, J. R. Stat. Soc. Ser. B Stat. Methodol., 71 (2009), pp. 1009-1030. · Zbl 1411.62107
[20] Ravikumar, P., Wainwright, M. J., Raskutti, G., and Yu, B., High-dimensional covariance estimation by minimizing \(\ell_1\) -penalized log-determinant divergence, Electron. J. Statist., 5 (2011), pp. 935-980. · Zbl 1274.62190
[21] Obozinski, G., Wainwright, M. J., and Jordan, M. I., Union support recovery in high-dimensional multivariate regression, in 2008 46th Annual Allerton Conference on Communication, Control, and Computing, , IEEE, Washington, DC, 2008, pp. 21-26. · Zbl 1373.62372
[22] Wang, W., Liang, Y., and Xing, E., Block regularized lasso for multivariate multi-response linear regression, in Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, , Scottsdale, AZ, 2013, pp. 608-617.
[23] Jalali, A., Sanghavi, S., Ruan, C., and Ravikumar, P. K., A dirty model for multi-task learning, in Advances in Neural Information Processing Systems, NeurIPS, San Diego, CA, 2010, pp. 964-972.
[24] Brunton, S. L., Proctor, J. L., and Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA, 113 (2016), pp. 3932-3937. · Zbl 1355.94013
[25] Cortiella, A., Park, K.-C., and Doostan, A., Sparse identification of nonlinear dynamical systems via reweighted \(\ell_1\) -regularized least squares, Comput. Methods Appl. Mech. Engrg., 376 (2021), 113620. · Zbl 1506.37104
[26] Kang, S. H., Liao, W., and Liu, Y., Ident: Identifying differential equations with numerical time evolution, J. Sci. Comput., 87 (2021), pp. 1-27. · Zbl 1467.65102
[27] Schaeffer, H., Learning partial differential equations via data discovery and sparse optimization, Proc. R. Soc. A Math. Phys. Eng. Sci., 473 (2017), 20160446. · Zbl 1404.35397
[28] Rudy, S. H., Brunton, S. L., Proctor, J. L., and Kutz, J. N., Data-driven discovery of partial differential equations, Sci. Adv., 3 (2017), e1602614.
[29] Schaeffer, H., Tran, G., and Ward, R., Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), pp. 3279-3295, doi:10.1137/18M116798X. · Zbl 1405.62127
[30] Chen, S. S., Donoho, D. L., and Saunders, M. A., Atomic decomposition by basis pursuit, SIAM Rev., 43 (2001), pp. 129-159, doi:10.1137/S003614450037906X. · Zbl 0979.94010
[31] Donoho, D. L. and Huo, X., Uncertainty principles and ideal atomic decomposition, IEEE Trans. Inform. Theory, 47 (2001), pp. 2845-2862. · Zbl 1019.94503
[32] Donoho, D. L., Elad, M., and Temlyakov, V. N., Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inform. Theory, 52 (2005), pp. 6-18. · Zbl 1288.94017
[33] Feuer, A. and Nemirovski, A., On sparse representation in pairs of bases, IEEE Trans. Inform. Theory, 49 (2003), pp. 1579-1581. · Zbl 1063.42018
[34] Candès, E. J. and Tao, T., Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), pp. 4203-4215. · Zbl 1264.94121
[35] Candès, E. J., Romberg, J., and Tao, T., Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509. · Zbl 1231.94017
[36] Knight, K. and Fu, W., Asymptotics for lasso-type estimators, Ann. Statist., 28 (2000), pp. 1356-1378. · Zbl 1105.62357
[37] Tropp, J. A., Just relax: Convex programming methods for identifying sparse signals in noise, IEEE Trans. Inform. Theory, 52 (2006), pp. 1030-1051. · Zbl 1288.94025
[38] Zhao, P. and Yu, B., On model selection consistency of lasso, J. Mach. Learn. Res., 7 (2006), pp. 2541-2563. · Zbl 1222.62008
[39] Fuchs, J.-J., Recovery of exact sparse representations in the presence of bounded noise, IEEE Trans. Inform. Theory, 51 (2005), pp. 3601-3608. · Zbl 1286.94031
[40] Meinshausen, N. and Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Statist., 34 (2006), pp. 1436-1462. · Zbl 1113.62082
[41] Ravikumar, P., Raskutti, G., Wainwright, M. J., and Yu, B., Model selection in Gaussian graphical models: High-dimensional consistency of \(\ell_1\) -regularized mle., in Advances in Neural Information Processing Systems 2008, NeurIPS, San Diego, CA, 2008, pp. 1329-1336.
[42] Fan, J. and Lv, J., A selective overview of variable selection in high dimensional feature space, Statist. Sinica, 20 (2010), pp. 101-148. · Zbl 1180.62080
[43] Mack, Y. and Silverman, B. W., Weak and strong uniform consistency of kernel regression estimates, Z. Wahrsch. Verw. Gebiete, 61 (1982), pp. 405-415. · Zbl 0495.62046
[44] Tusnády, G., A remark on the approximation of the sample DF in the multidimensional case, Period. Math. Hungar., 8 (1977), pp. 53-55. · Zbl 0386.60006
[45] Masry, E., Multivariate local polynomial regression for time series: Uniform strong consistency and rates, J. Time Ser. Anal., 17 (1996), pp. 571-599. · Zbl 0876.62075
[46] Li, Y. and Hsing, T., Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data, Ann. Statist., 38 (2010), pp. 3321-3351. · Zbl 1204.62067
[47] Silverman, B. W., Weak and strong uniform consistency of the kernel estimate of a density and its derivatives, Ann. Statist., 6 (1978), pp. 177-184. · Zbl 0376.62024
[48] Bonkile, M. P., Awasthi, A., Lakshmi, C., Mukundan, V., and Aswin, V. S., A systematic literature review of Burgers’ equation with recent advances, Pramana, 90 (2018), pp. 69.
[49] Rudenko, O. V. and Soluian, S. I., The Theoretical Principles of Nonlinear Acoustics, Moscow Izdatel Nauka, Moscow, Russia, 1975.
[50] Sawada, K. and Kotera, T., A method for finding n-soliton solutions of the KdV equation and KdV-like equation, Progr. Theoret. Phys., 51 (1974), pp. 1355-1367. · Zbl 1125.35400
[51] Boussinesq, J., Essai sur la théorie des eaux courantes, Impr. Nationale, Paris, France, 1877. · JFM 09.0680.04
[52] Audibert, J.-Y. and Tsybakov, A. B., Fast learning rates for plug-in classifiers, Ann. Statist., 35 (2007), pp. 608-633. · Zbl 1118.62041
[53] Ruppert, D., Sheather, S. J., and Wand, M. P., An effective bandwidth selector for local least squares regression, J. Amer. Stat. Assoc., 90 (1995), pp. 1257-1270. · Zbl 0868.62034
[54] Fan, J. and Lv, J., Sure independence screening for ultrahigh dimensional feature space, J. R. Stat. Soc. Ser. B Stat. Methodol., 70 (2008), pp. 849-911. · Zbl 1411.62187
[55] Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R., Least angle regression, Ann. Statist., 32 (2004), pp. 407-499. · Zbl 1091.62054
[56] Javanmard, A. and Montanari, A., Confidence intervals and hypothesis testing for high-dimensional regression, J. Mach. Learn. Res., 15 (2014), pp. 2869-2909. · Zbl 1319.62145
[57] He, Y., Kang, S. H., Liao, W., Liu, H., and Liu, Y., Robust PDE Identification from Noisy Data, preprint, https://arxiv.org/abs/2006.06557, 2020.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.