The paper studies second order expansions for the recently introduced class of nonsmooth functions with primal-dual gradient structure. For this class of lower semi-continuous and not necessarily convex functions it is possible to explicitly give a basis for the subspace
parallel to the Clarke subdifferential at some point. Relative to its orthogonal subspace
the function appears to be smooth, and it is actually possible to find smooth trajectories tangent to
along which the function is
. Along with this smooth restriction a smooth multiplier function can be defined. Having these two smooth objects at hand, a
Lagrangian is defined which leads to a second order expansion of the nonsmooth function along the subspace
. Explicit expressions for the first and second order derivatives are given. Connections between second order epi-derivatives and
-Hessians are made, and expressions for manifold restricted Hessians are given for partly smooth functions. A number of illuminating examples accompany the results of the article.