## Generalized differential calculus for nonsmooth and set-valued mappings.(English)Zbl 0807.49016

Some generalized differentiability concepts for multifunctions and nonsmooth mappings in $$\mathbb{R}^ n$$ are studied. In this paper the most important one is the so-called coderivative of multifunctions introduced earlier [the author, Dokl. Akad. Nauk SSSR 254, 1072-1076 (1980; Zbl 0491.49011)] using the normal cone to the graph in the sense given by the author in Prikl. Mat. Mekh. 40, 1014-1023 (1976; Zbl 0362.49017). This normal cone is not convex. The corresponding nonconvex coderivative appears to be useful in control theory to obtain necessary optimality conditions in differential inclusions. In this paper a rich calculus for the coderivative and related subdifferential constructions is developed using a variational approach, namely an extremal generalization of the separability theorem for nonconvex sets. The proof of this result is based on a special smooth approximation procedure using proximal points (the so-called metric approximation method).
Reviewer: V.Arnautu (Iaşi)

### MSC:

 49J52 Nonsmooth analysis 49K24 Optimal control problems with differential inclusions (nec./ suff.) (MSC2000)

### Citations:

Zbl 0491.49011; Zbl 0362.49017
Full Text: