## Local differentiability of distance functions.(English)Zbl 0960.49018

Let $$C$$ be a closed subset of a Hilbert space, and let $$x$$ be a prescribed point in the boundary of $$C$$. The authors discuss the continuous differentiability of the distance function $$\text{dist}[.;C]$$ outside of $$C$$ on some neighborhood of $$x$$. This paper is an important complement to a previous work by F. H. Clarke, R. J. Stern and P. R. Wolenski [J. Convex Anal. 2, No. 1-2, 117-144 (1995; Zbl 0881.49008)].
Reviewer: A.Seeger (Avignon)

### MSC:

 49J52 Nonsmooth analysis 58C20 Differentiation theory (Gateaux, Fréchet, etc.) on manifolds 90C30 Nonlinear programming 58C06 Set-valued and function-space-valued mappings on manifolds

Zbl 0881.49008
Full Text:

### References:

 [1] J. M. Borwein and J. R. Giles, The proximal normal formula in Banach space, Trans. Amer. Math. Soc. 302 (1987), no. 1, 371 – 381 (English, with French summary). · Zbl 0632.49007 [2] F. H. Clarke, R. J. Stern, and P. R. Wolenski, Proximal smoothness and the lower-\?² property, J. Convex Anal. 2 (1995), no. 1-2, 117 – 144. · Zbl 0881.49008 [3] Jean-Philippe Vial, Strong and weak convexity of sets and functions, Math. Oper. Res. 8 (1983), no. 2, 231 – 259. · Zbl 0526.90077 [4] Alexander Shapiro, Existence and differentiability of metric projections in Hilbert spaces, SIAM J. Optim. 4 (1994), no. 1, 130 – 141. · Zbl 0793.41021 [5] Herbert Federer, Curvature measures, Trans. Amer. Math. Soc. 93 (1959), 418 – 491. · Zbl 0089.38402 [6] R. A. Poliquin and R. T. Rockafellar, Prox-regular functions in variational analysis, Trans. Amer. Math. Soc. 348 (1996), no. 5, 1805 – 1838. · Zbl 0861.49015 [7] R. A. Poliquin and R. T. Rockafellar, Generalized Hessian properties of regularized nonsmooth functions, SIAM J. Optim. 6 (1996), no. 4, 1121 – 1137. · Zbl 0863.49010 [8] René Poliquin and Terry Rockafellar, Second-order nonsmooth analysis in nonlinear programming, Recent advances in nonsmooth optimization, World Sci. Publ., River Edge, NJ, 1995, pp. 322 – 349. · Zbl 0939.90022 [9] R. Tyrrell Rockafellar and Roger J.-B. Wets, Variational analysis, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317, Springer-Verlag, Berlin, 1998. · Zbl 0888.49001 [10] C. Combari, A. Elhilali Alaoui, A. Levy, R. Poliquin, and L. Thibault, Convex composite functions in Banach spaces and the primal lower-nice property, Proc. Amer. Math. Soc. 126 (1998), no. 12, 3701 – 3708. · Zbl 0918.58008 [11] René A. Poliquin, Integration of subdifferentials of nonconvex functions, Nonlinear Anal. 17 (1991), no. 4, 385 – 398. · Zbl 0831.49018 [12] René A. Poliquin, An extension of Attouch’s theorem and its application to second-order epi-differentiation of convexly composite functions, Trans. Amer. Math. Soc. 332 (1992), no. 2, 861 – 874. · Zbl 0794.49014 [13] A. B. Levy, R. Poliquin, and L. Thibault, Partial extensions of Attouch’s theorem with applications to proto-derivatives of subgradient mappings, Trans. Amer. Math. Soc. 347 (1995), no. 4, 1269 – 1294. · Zbl 0837.49013 [14] Lionel Thibault and Dariusz Zagrodny, Integration of subdifferentials of lower semicontinuous functions on Banach spaces, J. Math. Anal. Appl. 189 (1995), no. 1, 33 – 58. · Zbl 0826.49009 [15] R. Correa, A. Jofré, and L. Thibault, Characterization of lower semicontinuous convex functions, Proc. Amer. Math. Soc. 116 (1992), no. 1, 67 – 72. · Zbl 0762.49007 [16] Edgar Asplund, Čebyšev sets in Hilbert space, Trans. Amer. Math. Soc. 144 (1969), 235 – 240. [17] Jean-Pierre Aubin and Ivar Ekeland, Applied nonlinear analysis, Pure and Applied Mathematics (New York), John Wiley & Sons, Inc., New York, 1984. A Wiley-Interscience Publication. · Zbl 0641.47066 [18] Ka Sing Lau, Almost Chebyshev subsets in reflexive Banach spaces, Indiana Univ. Math. J. 27 (1978), no. 5, 791 – 795. · Zbl 0398.41026 [19] J. M. Borwein and H. M. Strójwas, Proximal analysis and boundaries of closed sets in Banach space. II. Applications, Canad. J. Math. 39 (1987), no. 2, 428 – 472. · Zbl 0621.46019 [20] Frank H. Clarke, Optimization and nonsmooth analysis, Canadian Mathematical Society Series of Monographs and Advanced Texts, John Wiley & Sons, Inc., New York, 1983. A Wiley-Interscience Publication. F. H. Clarke, Optimization and nonsmooth analysis, 2nd ed., Classics in Applied Mathematics, vol. 5, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1990. · Zbl 0582.49001 [21] E. Asplund and R. T. Rockafellar, Gradients of convex functions, Trans. Amer. Math. Soc. 139 (1969), 443 – 467. · Zbl 0181.41901 [22] T. S. Motzkin, Sur quelques propriétés caractéristiques des ensembles convexes, Att. R. Acad. Lincei, Rend. 21 (1935), 562-567. · Zbl 0011.41105 [23] Victor Klee, Convexity of Chevyshev sets, Math. Ann. 142 (1960/1961), 292 – 304. · Zbl 0091.27701 [24] Jean-Baptiste Hiriart-Urruty, Ensembles de Tchebychev vs. ensembles convexes: l’état de la situation vu via l’analyse convexe non lisse, Ann. Sci. Math. Québec 22 (1998), no. 1, 47 – 62 (French, with English and French summaries). · Zbl 1098.49506 [25] Robert R. Phelps, Convex functions, monotone operators and differentiability, 2nd ed., Lecture Notes in Mathematics, vol. 1364, Springer-Verlag, Berlin, 1993. · Zbl 0921.46039
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.