In this paper the authors consider the following second-oder Hamiltonian system
where , and . A nonzero solution of (1) is said to be homoclinic (to 0) if as .
The authors deal with the existence and multiplicity of homoclinic solutions under the assumption that is indefinite sign and subquadratic as by using the genus properties in critical theory. The results in this paper extends the results in [Z. Zhang and R. Yuan, Nonlinear Anal., Theory Methods Appl. 71, No. 9, A, 4125–4130 (2009; Zbl 1173.34330)] and [J. Sun, H. Chen and J. J. Nieto, J. Math. Anal. Appl. 373, No. 1, 20–29 (2011; Zbl 1230.37079)]. The main results are the following.
Theorem 1.1 Assume that and satisfy the following conditions:
is a definite symmetric matrix for all and there exists a constant such that
for all ;
There exist two constants and two functions such that
for all , and
for all ;
There exist two functions and such that
for all , where as ;
There exist an open set and two constants and such that
for all , .
Then Problem (1) has at least one nontrivial homoclinic orbit.
Theorem 1.2 Assume that and satisfy (L), (), (), () and that for all and . Then Problem (1) has infinitely many homoclinic orbits.