A finite-horizon $H_\infty$ state-feedback control problem for singularly-perturbed linear time-dependent systems with a small state delay is considered. Two approaches to the asymptotic analysis and solution of this problem are proposed. In the first one, an asymptotic solution of the singularly perturbed system of functional-differential equations of Riccati type, associated with the original $H_\infty$ problem via sufficient conditions on the existence of its solution, is constructed. Based on this asymptotic solution, conditions for the existence of a solution of the original $H_\infty$ problem, independent of the small parameter of the singular perturbations, are derived. A simplified controller problem for all sufficiently small values of this parameter is obtained. In the second approach, the original $H_\infty$ problem is decomposed into two lower-dimensional parameter-independent $H_\infty$ subproblems, the reduced-order (slow) and the boundary-layer (fast) subproblems; controllers solving these subproblems are constructed. Based on these controllers, a composite controller is derived, which solves the original $H_\infty$ problem for all sufficiently small values of the singular perturbation parameter. An illustrative example is presented.