Let

$M$ be a complete Riemannian

$n$-manifold of bounded curvature. For any

$\epsilon >0$, the

$\epsilon $-collapsed part of

$M$ is defined as the set

${\mathcal{C}}^{n}\left(\epsilon \right)$ of points at which the injectivity radius of the exponential map is

$<\epsilon $. The authors study the structure of the

$\epsilon $-collapsed part of a manifold

$M$ for suitably small

$\epsilon $. The main results show that the local geometry of

${\mathcal{C}}^{n}\left(\epsilon \right)$ is encoded partially in the symmetry properties of a nearby metric. More precisely, a given metric can be closely approximated by one that admits a sheaf of nilpotent Lie algebras of local Killing vector fields pointing in all sufficiently collapsed directions of

${\mathcal{C}}^{n}\left(\epsilon \right)$. This sheaf is called the nilpotent Killing structure. A detailed construction of this nilpotent Killing structure is presented, its properties are established and some applications to the description of collapses of a Riemannian manifold

$M$ are indicated.