×

zbMATH — the first resource for mathematics

Sequential quadratic optimization for nonlinear equality constrained stochastic optimization. (English) Zbl 07351608
MSC:
90C30 Nonlinear programming
90C55 Methods of successive quadratic programming type
65K05 Numerical mathematical programming methods
65K10 Numerical optimization and variational techniques
90C15 Stochastic programming
Software:
CUTEr; CUTE ; Ipopt; LANCELOT; SOCS
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] D. P. Bertsekas, Network Optimization: Continuous and Discrete Models, Athena Scientific Belmont, MA, 1998. · Zbl 0997.90505
[2] J. T. Betts, Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, SIAM, Philadelphia, 2010. · Zbl 1189.49001
[3] I. Bongartz, A. R. Conn, N. Gould, and P. L. Toint, Cute: Constrained and unconstrained testing environment, ACM Trans. Math. Software, 21 (1995), pp. 123-160. · Zbl 0886.65058
[4] L. Bottou, F. E. Curtis, and J. Nocedal, Optimization methods for large-scale machine learning, SIAM Rev., 60 (2018), pp. 223-311. · Zbl 1397.65085
[5] R. H. Byrd, F. E. Curtis, and J. Nocedal, An inexact SQP method for equality constrained optimization, SIAM J. Optim., 19 (2008), pp. 351-369. · Zbl 1158.49035
[6] R. H. Byrd, F. E. Curtis, and J. Nocedal, An inexact Newton method for nonconvex equality constrained optimization, Math. Program., 122 (2010), pp. 273-299. · Zbl 1184.90127
[7] R. H. Byrd, J. C. Gilbert, and J. Nocedal, A trust region method based on interior point techniques for nonlinear programming, Math. Program., 89 (2000), pp. 149-185. · Zbl 1033.90152
[8] R. H. Byrd, M. E. Hribar, and J. Nocedal, An interior point algorithm for large-scale nonlinear programming, SIAM J. Optim., 9 (1999), pp. 877-900. · Zbl 0957.65057
[9] C. Chen, F. Tung, N. Vedula, and G. Mori, Constraint-aware deep neural network compression, in Proceedings of the ECCV, 2018, pp. 400-415.
[10] A. R. Conn, N. I. M. Gould, and P. L. Toint, LANCELOT: A Fortran Package for Large-Scale Nonlinear Optimization, Springer, New York, 1992. · Zbl 0761.90087
[11] R. Courant, Variational methods for the solution of problems of equilibrium and vibrations, Bull. Amer. Math. Soc., 49 (1943), pp. 1-23. · Zbl 0063.00985
[12] F. E. Curtis and D. P. Robinson, Exploiting negative curvature in deterministic and stochastic optimization, Math. Program. Ser. B, 176 (2019), pp. 69-94. · Zbl 1417.49036
[13] E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), pp. 201-213. · Zbl 1049.90004
[14] R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, Chichester, UK, 1987. · Zbl 0905.65002
[15] S. P. Han, A globally convergent method for nonlinear programming, J. Optim. Theory Appl., 22 (1977), pp. 297-309. · Zbl 0336.90046
[16] S. P. Han and O. L. Mangasarian, Exact penalty functions in nonlinear programming, Math. Program, 17 (1979), pp. 251-269. · Zbl 0424.90057
[17] E. Hazan and H. Luo, Variance-reduced and projection-free stochastic optimization, in Proceedings of the International Conference on Machine Learning, 2016, pp. 1263-1271.
[18] M. R. Hestenes, Multiplier and Gradient Methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320. · Zbl 0174.20705
[19] S. Kumar Roy, Z. Mhammedi, and M. Harandi, Geometry aware constrained optimization techniques for deep learning, in Proceedings of CVPR, 2018, pp. 4460-4469.
[20] F. Kupfer and E. W. Sachs, Numerical solution of a nonlinear parabolic control problem by a reduced SQP method, Comput. Optim. Appl., 1 (1992), pp. 113-135. · Zbl 0771.49010
[21] F. Locatello, A. Yurtsever, O. Fercoq, and V. Cevher, Stochastic Frank-Wolfe for composite convex minimization, in Proceedings of NeurIPS, 2019, pp. 14269-14279.
[22] H. Lu and R. M. Freund, Generalized stochastic Frank-Wolfe algorithm with stochastic “substitute” gradient for structured convex optimization, Math. Program., 187 (2021), pp. 317-349. · Zbl 1465.90063
[23] Y. Nandwani, A. Pathak, and P. Singla, A primal-dual formulation for deep learning with constraints, in Proceedings of NeurIPS, 2019, pp. 12157-12168.
[24] Y. Nesterov, Introductory Lectures on Convex Optimization, Appl. Optim., Springer, New York, 2004.
[25] J. Nocedal and S. Wright, Numerical Optimization, Springer Ser. Oper. Res. Financ. Eng., Springer, New York, 2006.
[26] M. J. D. Powell, A Method for Nonlinear Constraints in Minimization Problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 283-298.
[27] M. J. D. Powell, A fast algorithm for nonlinearly constrained optimization calculations, in Numerical Analysis, Lecture Notes in Math., Springer, New York, 1978, pp. 144-157.
[28] S. N. Ravi, T. Dinh, V. S. Lokhande, and V. Singh, Explicitly imposing constraints in deep networks via conditional gradients gives improved generalization and faster convergence, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 4772-4779.
[29] S. J. Reddi, S. Sra, B. Póczos, and A. Smola, Stochastic Frank-Wolfe methods for nonconvex optimization, in Proceedings of the 54th Annual Allerton Conference, IEEE, 2016, pp. 1244-1251.
[30] T. Rees, H. S. Dollar, and A. J. Wathen, Optimal solvers for pde-constrained optimization, SIAM J. Sci. Comput., 32 (2010), pp. 271-298. · Zbl 1208.49035
[31] A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, SIAM, Philadelphia, 2009. · Zbl 1183.90005
[32] Y. L. Tong, The Multivariate Normal Distribution, Springer, New York, 2012.
[33] A. Wächter and L. T. Biegler, Line search filter methods for nonlinear programming: Motivation and global convergence, SIAM J. Optim., 16 (2005), pp. 1-31. · Zbl 1114.90128
[34] A. Waechter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program., 106 (2006), pp. 25-57. · Zbl 1134.90542
[35] R. B. Wilson, A Simplicial Algorithm for Concave Programming, Ph.D. thesis, Graduate School of Business Administration, Harvard University, Cambridge, MA, 1963.
[36] M. Zhang, Z. Shen, A. Mokhtari, H. Hassani, and A. Karbasi, One sample stochastic Frank-Wolfe, in Proceedings of AISTATS, 2020, pp. 4012-4023.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.