Sign-changing solutions to Schrödinger-Kirchhoff-type equations with critical exponent

Abstract

In this paper, we study the following Schrödinger-Kirchhoff-type equations:

$$ \textstyle\begin{cases}-(a+b\int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx)\triangle u+u= k(x)|u|^{2^{*}-2}u+\mu h(x)u \quad\mbox{in } \mathrm{R}^{3},\\ u\in H^{1}(\mathrm{R}^{3}), \end{cases} $$

where $a, b,\mu>0$ are constants, $2^{*}=6$ is the critical Sobolev exponent in three spatial dimensions. Under appropriate assumptions on nonnegative functions $k(x)$ and $h(x)$, we establish the existence of positive and sign-changing solutions by variational methods.

1 Introduction

In this paper, we investigate the following Schrödinger-Kirchhoff-type problem:

$$ \textstyle\begin{cases} -(a+b\int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx)\triangle u+u= k(x)|u|^{2^{*}-2}u+\mu h(x)u \quad\mbox{in } \mathrm{R}^{3},\\ u\in H^{1}(\mathrm{R}^{3}), \end{cases} $$

(1.1)

where $a, b>0$ are constants, $2^{*}=6$ is the critical Sobolev exponent in dimension three. We assume μ, functions $k(x)$ and $h(x)$ satisfy the following hypotheses:

($\mu_{1}$):

$0<\mu<\tilde{\mu}$, where μ̃ is defined by

$$\tilde{\mu}:=\inf_{u\in H^{1}(\mathrm{R}^{3})\setminus\{0\}} \biggl\{ \int_{\mathrm{R}^{3}}\bigl(a|\nabla u|^{2}+|u|^{2} \bigr)\,dx: \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx=1\biggr\} ; $$

(k₁):

$k(x)\geq0$, $\forall x\in\mathrm{R}^{3}$;

(k₂):

there exist $x_{0}\in\mathrm{R}^{3}$, $\sigma_{1}>0$, $\rho_{1}>0$, and $1\leq \alpha<3$ such that $k(x_{0})=\max_{x\in\mathrm{R}^{3}}k(x)$ and

$$\bigl|k(x)-k(x_{0})\bigr|\leq\sigma_{1}|x-x_{0}|^{\alpha}\quad\mbox{for } |x-x_{0}|< \rho_{1}; $$

(h₁):

$h(x)\geq0$ for any $x\in R^{3}$ and $h(x)\in L^{\frac {3}{2}}(\mathrm{R}^{3})$;

(h₂):

there exist $\sigma_{2}>0$ and $\rho_{2}>0$ such that $h(x)\geq \sigma_{2}|x-x_{0}|^{-\beta}$ for $|x-x_{0}|<\rho_{2}$.

The Kirchhoff-type problem is related to the stationary analog of the equation

$$u_{tt}-\biggl(a+b \int_{\Omega}|\nabla u|^{2}\,dx\biggr)\triangle u= f(x,u) \quad\mbox{in } \Omega, $$

where Ω is a bounded domain in $\mathrm{R}^{N}$, u denotes the displacement, $f(x,u)$ the external force and b the initial tension while a is related to the intrinsic properties of the string (such as Young’s modulus). Equations of this type arise in the study of string or membrane vibration and were proposed by Kirchhoff in 1883 (see [1]) to describe the transversal oscillations of a stretched string, particularly, taking into account the subsequent change in string length caused by oscillations.

Kirchhoff-type problems are often referred to as being nonlocal because of the presence of the integral over the entire domain Ω, which provokes some mathematical difficulties. Similar nonlocal problems also model several physical and biological systems where u describes a process which depends on the average of itself, for example, the population density; see [2, 3]. Kirchhoff-type problems have received much attention. Some important and interesting results can be found; see, for example, [4–6] and the references therein.

The solvability of the following Schrödinger-Kirchoff-type equation (1.2) has also been well studied in general dimensions by various authors:

$$ -\biggl(a+b \int_{\mathrm{R}^{N}}|\nabla u|^{2}\,dx\biggr)\triangle u+V(x)u=f(x,u)\quad \mbox{in } \mathrm{R}^{N}. $$

(1.2)

For example, Wu [7] and many others [8–13], using variational methods, proved the existence of nontrivial solutions to (1.2) with subcritical nonlinearities. Li and Ye [14] obtained the existence of positive solution for (1.2) with critical exponents. More recently, Wang et al. [15] and other author [16] proved the existence and multiplicity of positive solutions of (1.2) with critical growth and a small positive parameters.

The problem of finding sign-changing solutions is a very classical problem. In general, this problem is much more difficult than finding a mere solution. There were several abstract theories or methods to study sign-changing solutions; see for example [17, 18] and the references therein. In recent years, Zhang and Perera [19] obtained sign-changing solutions of (1.2) with superlinear or asymptotically linear terms. More recently, Mao and Zhang [20] use minimax methods and invariant sets of descent flow to prove the existence of nontrivial solutions and sign-changing solutions for (1.2) without the P.S. condition. Motivated by the above works, in this paper our aim is to study the existence of positive and sign-changing solutions for the problem (1.1). The method is inspired by Hirano and Shioji [21] and Huang et al. [22]; however, the argument used by them cannot be directly applied here. To the best of our knowledge, there are very few works up to now studying sign-changing solutions for Schrödinger-Kirchhoff-type problem with critical exponent, i.e. the problem (1.1). Our main results are as follows.

Theorem 1.1

Assume that ($\mu_{1}$), (k₁), (k₂), and (h₁)-(h₂) hold, then for $1<\beta<3$, the problem (1.1) possesses at least one positive solution.

Theorem 1.2

Assume ($\mu_{1}$), (k₁), (k₂), and (h₁)-(h₂) hold, then for $\frac{3}{2}<\beta<3$, the problem (1.1) possesses at least one sign-changing solution.

Notations

• $H^{1}(\mathrm{R}^{3})$ is the Sobolev space equipped with the norm $\|u\|^{2}_{H^{1}(\mathrm{R}^{3})}=\int_{\mathrm{R}^{3}}{(|\nabla u|^{2}+|u|^{2})\,dx}$.

• Define $\|u\|^{2}:=\int_{\mathrm{R}^{3}}{(a|\nabla u|^{2}+|u|^{2})\,dx}$ for $u\in H^{1}(\mathrm{R}^{3})$. Note that $\|\cdot\|$ is an equivalent norm on $H^{1}(\mathrm{R}^{3})$.

• For any $1\leq s\leq\infty$, $\|u\|_{L^{s}}:=(\int_{\mathrm{R}^{3}}|u|^{s} \,dx)^{\frac{1}{s}}$ denotes the usual norm of the Lebesgue space $L^{s}(\mathrm{R}^{3})$.

• Let $D^{1,2}(\mathrm{R}^{3})$ is the completion of $C_{0}^{\infty}(\mathrm{R}^{3})$ with respect to the norm $\|u\|^{2}_{D^{1,2}(\mathrm{R}^{3})}:=\int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx$.

• S denotes the best Sobolev constant defined by $S=\inf_{u\in D^{1,2}(\mathrm{R}^{3})\setminus\{0\}}\frac{\int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx}{(\int_{\mathrm{R}^{3}} u^{6} \,dx)^{\frac{1}{3}}}$.

• $C>0$ denotes various positive constants.

The outline of the paper is given as follows: in Section 2, we present some preliminary results. In Sections 3 and 4, we give the proofs of Theorems 1.1 and 1.2, respectively.

2 The variational framework and preliminary

In this section, we give some preliminary lemmas and the variational setting for (1.1). It is clear that system (1.1) is the Euler-Lagrange equations of the functional $I:H^{1}(\mathrm{R}^{3})\rightarrow\mathrm{R}$ defined by

$$ I(u)=\frac{1}{2}\|u\|^{2}+\frac{b}{4}\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2}- \frac{1}{6} \int_{\mathrm{R}^{3}}k(x)|u|^{6}\,dx-\frac{\mu }{2} \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx. $$

(2.1)

Obviously, I is a well-defined $C^{1}$ functional and satisfies

$$\begin{aligned} \bigl\langle I'(u),v\bigr\rangle ={}& \int_{\mathrm{R}^{3}}{(a\nabla u\nabla v+uv)}\,dx+b \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx \int_{\mathrm{R}^{3}}\nabla u\nabla v\,dx \\ &{}- \int_{\mathrm{R}^{3}}\bigl({k(x)|u|^{4}uv}+\mu h(x)uv\bigr)\,dx, \end{aligned}$$

(2.2)

for $v\in H^{1}(\mathrm{R}^{3})$. It is well known that $u\in H^{1}(\mathrm{R}^{3})$ is a critical point of the functional I if and only if u is a weak solution of (1.1).

Lemma 2.1

Assume (h₁) holds. Then the functions $\psi_{h}:u\in H^{1}(\mathrm{R}^{3})\mapsto\int_{\mathrm{R}^{3}} h(x)u^{2}\,dx$ is weakly continuous. And for each $v\in H^{1}(\mathrm{R}^{3})$, $\varphi_{h}:u\in H^{1}(\mathrm{R}^{3})\mapsto\int_{\mathrm{R}^{3}}h(x)uv\,dx$ is also weakly continuous.

The proof of Lemma 2.1 is a direct conclusion of [23], Lemma 2.13.

Lemma 2.2

Assume (h₁) holds. Then the infimum μ̃ is achieved

$$\tilde{\mu}:=\inf_{u\in H^{1}(\mathrm{R}^{3})\setminus\{ 0\}} \biggl\{ \int_{\mathrm{R}^{3}}\bigl(a|\nabla u|^{2}+|u|^{2} \bigr)\,dx: \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx=1\biggr\} . $$

Proof

The proof of Lemma 2.2 is the same as [24], Lemma 2.5; here we omit it for simplicity. □

Lemma 2.3

Assume (k₁), (h₁), and ($\mu_{1}$) hold. Then the functional I exhibits the following properties.

• (1) There exist $\rho, \gamma>0$ such that $I(u)\geq\gamma$ for $\|u\| =\rho$.

• (2) There exists $e\in H^{1}(\mathrm{R}^{3})$ with $\|e\|>\rho$ such that $I(e)<0$.

Proof

By Lemma 2.2 and the Sobolev inequality, we obtain

$$I(u)\geq\frac{1}{2}\|u\|^{2}-C\|u\|^{6}- \frac{\mu}{2\tilde{\mu}}\|u\|^{2}=\| u\|^{2}\biggl( \frac{1}{2}-\frac{\mu}{2\tilde{\mu}}-C\|u\|^{4}\biggr). $$

Set $\|u\|=\rho$ small enough such that $C\rho^{4}\leq\frac{1}{4}(1-\frac {\mu}{\tilde{\mu}})$, then we have

$$ I(u)\geq\frac{1}{4}\biggl(1-\frac{\mu}{\tilde{\mu}}\biggr)\rho^{2}. $$

(2.3)

Choosing $\gamma=\frac{1}{4}(1-\frac{\mu}{\tilde{\mu}})\rho^{2}$, we complete the proof of (1).

For $t>0$ and some $u_{0}\in H^{1}(\mathrm{R}^{3})$ with $\|u_{0}\|=1$, it follows from (h₁) and ($\mu_{1}$) that

$$I(tu_{0})\leq\frac{1}{2}t^{2}\|u_{0} \|^{2}+\frac{b}{4}t^{4}\biggl( \int_{\mathrm{R}^{3}}|\nabla u_{0}|^{2}\,dx \biggr)^{2} -\frac{t^{6}}{6} \int_{\mathrm{R}^{3}}k(x)|u_{0}|^{6}\,dx, $$

which implies that $I(tu_{0})<0$ for $t>0$ large enough. Hence, we can take an $e=t_{1}u_{0}$ for some $t_{1}>0$ large enough and (2) follows. □

Next, we define the Nehari manifold N associated with I

$$N:=\bigl\{ u\in H^{1}\bigl(\mathrm{R}^{3}\bigr)\backslash\{0 \}:G(u)=0\bigr\} , \quad\mbox{where } G(u)=\bigl\langle I'(u),u\bigr\rangle . $$

Now we state some properties of N.

Lemma 2.4

Assume ($\mu_{1}$) satisfies, then the following conclusions hold.

• (1) For all $u\in H^{1}(\mathrm{R}^{3})\backslash\{0\}$, there exists a unique $t(u)>0$ such that $t(u)u\in N$. Moreover, $I(t(u))u= \max_{t\geq0}I(tu)$.

• (2) $0< t(u)<1$ in the case $\langle I'(u),u\rangle<0$; $t(u)>1$ in the case $\langle I'(u),u\rangle>0$.

• (3) $t(u)$ is a continuous functional with respect to u in $H^{1}(\mathrm{R}^{3})$.

• (4) $t(u)\rightarrow+\infty$ as $\|u\|\rightarrow0$.

Proof

The proof is similar to that of [22], Lemma 2.4, and is omitted here. □

3 Positive solution

In order to deduce Theorem 1.1, the following lemmas are important. Borrowing an idea from Lemma 3.6 in [14], we can obtain the first result.

Lemma 3.1

For $s, t>0$, the system

$$\begin{aligned} \left \{ \textstyle\begin{array}{@{}l} f(t,s)=t-aS(\frac{s+t}{\lambda})^{\frac{1}{3}}=0,\\ g(t,s)=s-bS^{2}(\frac{s+t}{\lambda})^{\frac{2}{3}}=0, \end{array}\displaystyle \right . \end{aligned}$$

has a unique solution $(t_{0},s_{0})$, where $\lambda>0$ is a constant. Moreover, if

$$\begin{aligned} \left \{ \textstyle\begin{array}{@{}l} f(t,s)\geq0,\\ g(t,s)\geq0, \end{array}\displaystyle \right . \end{aligned}$$

then $t\geq t_{0}$ and $s\geq s_{0}$, where $t_{0}=\frac{abS^{3}+a\sqrt {b^{2}S^{6}+4\lambda aS^{3}}}{2\lambda}$, $s_{0}=\frac{bS^{6}+2\lambda abS^{3}+b^{2}S^{3}\sqrt{b^{3}S^{6}+4\lambda aS^{3}}}{2\lambda^{2}}$.

Lemma 3.2

Assume ($\mu_{1}$), (k₁), and (h₁) hold. Let sequence $\{u_{n}\}\subset N$ be such that $u_{n}\rightharpoonup u$ in $H^{1}(\mathrm{R}^{3})$ and $I(u_{n})\rightarrow c$, but any subsequence of $\{ u_{n}\}$ does not converge strongly to u. Then one of the following results holds:

• (1) $c>I(t(u)u)$ in the case $u\neq0$ and $\langle I'(u),u\rangle<0$;

• (2) $c\geq c^{*}$ in the case $u=0$;

• (3) $c>c^{*}$ in the case $u\neq0$ and $\langle I'(u),u\rangle\geq0$;

where $c^{*}=\frac{abS^{3}}{4\|k\|_{\infty}}+\frac{b^{3}S^{6}}{24\|k\|^{2}_{\infty}}+\frac{(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac{3}{2}}}{24\|k\|^{2}_{\infty}}$, $t(u)$ is defined as in Lemma  2.4.

Proof

Part of the proof is similar to that of [22], Lemma 3.1, or [25], Proposition 3.3. For the reader’s convenience, we sketch the proof here briefly. Since $u_{n}\rightharpoonup u$ in $H^{1}(\mathrm{R}^{3})$, we have $u_{n}-u\rightharpoonup0$. Then by Lemma 2.1, we obtain

$$ \int_{\mathrm{R}^{3}}h(x)|u_{n}-u|^{2}\,dx\rightarrow0. $$

(3.1)

We obtain from the Brézis-Lieb lemma [26], (3.1), and $u_{n}\in N$

$$\begin{aligned} c+o(1)={}&I(u_{n}) =I(u)+\frac{1}{2}\|u_{n}-u\|^{2}+\frac{b}{4} \biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(u_{n}-u)\bigr|^{2}\,dx \biggr)^{2} \\ &{}-\frac{1}{6} \int_{\mathrm{R}^{3}}k(x)|u_{n}-u|^{6}\,dx+o(1) \end{aligned}$$

(3.2)

and

$$\begin{aligned} 0={}&\bigl\langle I'(u_{n}),u_{n} \bigr\rangle =\bigl\langle I'(u),u\bigr\rangle +\|u_{n}-u \|^{2}+b \biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla (u_{n}-u)\bigr|^{2}\,dx \biggr)^{2} \\ &{}- \int_{\mathrm{R}^{3}}k(x)|u_{n}-u|^{6}\,dx+o(1). \end{aligned}$$

(3.3)

Up to a subsequence, we may assume that there exists $l_{i}\geq 0$, $i=1,2,3$ such that

$$ \begin{aligned} &\|u_{n}-u\|^{2}\rightarrow l_{1},\qquad b \biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla (u_{n}-u)\bigr|^{2}\,dx \biggr)^{2}\rightarrow l_{2}, \\ &\int_{\mathrm{R}^{3}}k(x)|u_{n}-u|^{6}\,dx\rightarrow l_{3}. \end{aligned} $$

(3.4)

Since any subsequence of $\{u_{n}\}$ does not converge strongly to u, one has $l_{1}>0$. Set $\gamma(t)=\frac{l_{1}}{2}t^{2}+\frac{l_{2}}{4}t^{4}-\frac {l_{3}}{6}t^{6}$ and $\eta(t)=g(t)+\gamma(t)$. By (3.3) and (3.4), we have $\eta' (1)=g'(1)+\gamma'(1)=0$ and $t=1$ is the only critical point of $\eta (t)$ in $(0,+\infty)$, which implies that

$$ \eta(1)=\max_{t>0}\eta(t). $$

(3.5)

We consider three situations:

(1) When $u\neq0$ and $\langle I'(u),u\rangle<0$, then by (3.3) and (3.4) we have

$$ l_{1}+l_{2}-l_{3}>0. $$

(3.6)

Then

$$ \gamma'(t)=l_{1}t+l_{2}t^{3}-l_{3}t^{5}>l_{1}t+l_{2}t^{3}-(l_{1}+l_{2})t^{5}= \bigl(1-t^{2}\bigr)\bigl[l_{1}t+(l_{1}+l_{2})t^{3} \bigr]\geq0 $$

(3.7)

for any $0< t<1$, which implies that

$$ \gamma(t)>\gamma(0)=0 \quad\mbox{for any } t\in(0,1). $$

(3.8)

Since $\langle I'(u),u\rangle<0$, by Lemma 2.4 there exists a $t(u)>0$ such that $0< t(u)<1$. Then it follows from (3.8) that $\gamma(t(u))>0$. Therefore, we obtain from (3.2) and (3.5) that $c=\eta(1)>\eta (t(u))=g(t(u))+\gamma(t(u))>I(t(u)u)$, which implies (1) holds.

(2) When $u=0$, then by (3.2), (3.3), and (3.4) we get

$$ \textstyle\begin{cases} l_{1}+l_{2}-l_{3}=0,\\ \frac{1}{2}l_{1}+\frac{1}{4}l_{2}-\frac{1}{6}l_{3}=c. \end{cases} $$

By the definition of S, we see that

$$\begin{aligned} & \int_{\mathrm{R}^{3}}|\nabla u_{n}|^{2}\,dx\geq \frac{S}{\|k\|_{\infty}^{1/3}} \biggl( \int_{\mathrm{R}^{3}}k(x)|u_{n}|^{6}\,dx \biggr)^{\frac{1}{3}}, \\ &b \biggl( \int_{\mathrm{R}^{3}}|\nabla u_{n}|^{2}\,dx \biggr)^{2}\geq b\frac{ S^{2}}{\|k\| _{\infty}^{2/3}} \biggl( \int_{\mathrm{R}^{3}}k(x)|u_{n}|^{6}\,dx \biggr)^{\frac{2}{3}}. \end{aligned}$$

Then

$$l_{1}\geq a S\biggl(\frac{l_{1}+l_{2}}{\|k\|_{\infty}}\biggr)^{\frac {1}{3}} \quad \mbox{and}\quad l_{2}\geq b S^{2}\biggl(\frac{l_{1}+l_{2}}{\|k\|_{\infty}} \biggr)^{\frac{2}{3}}. $$

Obviously, if $l_{1}>0$, then $l_{2}, l_{3}>0$. It follows from Lemma 3.1 that

$$\begin{aligned} c&=\frac{1}{3}l_{1}+\frac{1}{12}l_{2} \\ &\geq\frac{1}{3}\frac{abS^{3}+a\sqrt{b^{2}S^{6}+4\|k\|_{\infty}aS^{3}}}{2\|k\| _{\infty}}+\frac{1}{12}\frac{bS^{6}+2\|k\|_{\infty}abS^{3}+b^{2}S^{3}\sqrt {b^{3}S^{6}+4\|k\|_{\infty}aS^{3}}}{ 2\|k\|_{\infty}^{2}} \\ &=\frac{abS^{3}}{4\|k\|_{\infty}}+\frac{b^{3}S^{6}}{24\|k\|_{\infty}^{2}}+\frac {(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac{3}{2}}}{24\|k\|_{\infty}^{2}}:=c^{*}. \end{aligned}$$

(3.9)

(3) When $u\neq0$ and $\langle I'(u),u\rangle\geq0$, we prove this case in two steps. First of all, we consider $u\neq0$ and $\langle I'(u),u\rangle=0$. Then from Lemma 2.3 and Lemma 2.4 we get

$$ I(u)=\max_{t>0}I(tu)>0. $$

(3.10)

Since $u\neq0$ and $\langle I'(u),u\rangle=0$, by the same process as (3.9) we obtain

$$ c=\eta(1)=I(u)+\frac{l_{1}}{3}+\frac{l_{2}}{12}>c^{*}. $$

(3.11)

Second, we prove the case $u\neq0$ and $\langle I'(u),u\rangle>0$. Set $t^{**}=(\frac{l_{2}+\sqrt{l_{2}^{2}+4l_{1}l_{3}}}{2l_{3}})^{\frac{1}{2}}$. Then $\gamma(t)$ attains its maximum at $t^{**}$, i.e.,

$$\begin{aligned} \gamma\bigl(t^{**}\bigr)&= \max_{t>0} \gamma(t) \\ &=\frac{l_{1}l_{2}}{4l_{3}}+\frac{l_{2}^{2}}{24l_{3}^{2}}+\frac {(l_{2}^{2}+4l_{1}l_{3})^{\frac{3}{2}}}{24l_{3}^{2}} \\ &\geq\frac{abS^{3}}{4\|k\|_{\infty}}+\frac{b^{3}S^{6}}{24\|k\|_{\infty}^{2}}+\frac{(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac{3}{2}}}{24\|k\|_{\infty}^{2}}=c^{*}. \end{aligned}$$

(3.12)

It follows from Lemma 2.4 that $0< t^{**}<1$. Then $I(t^{**}u)\geq0$. Therefore, by (3.2), (3.5), and (3.12) we obtain

$$ c=\eta(1)>\eta\bigl(t^{**}\bigr)=I\bigl(t^{**}u \bigr)+\gamma\bigl(t^{**}\bigr)\geq c^{*}. $$

The proof of Lemma 3.2 is complete. □

Lemma 3.3

If the hypotheses of Theorem  1.1 hold with $1<\beta <3$, then

$$c_{1}< \frac{abS^{3}}{4\|k\|_{\infty}}+\frac{b^{3}S^{6}}{24\|k\|_{\infty}^{2}}+\frac {(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac{3}{2}}}{24\|k\|_{\infty}^{2}}=c^{*}, $$

where $c_{1}$ is defined by $\inf_{u\in N}I(u)$.

Proof

We borrow from an idea employed in [22] to prove this lemma. For $\varepsilon,r>0$, define $w_{\varepsilon}(x)=\frac{C\varphi (x)\varepsilon^{\frac{1}{4}}}{(\varepsilon+|x-x_{0}|^{2})^{\frac{1}{2}}}$, where C is a normalizing constant, $x_{0}$ is given in (k₂) and $\varphi\in C_{0}^{\infty}(R^{3})$, $0\leq\varphi\leq1$, $\varphi |_{B_{r}(0)}\equiv1$, and $\operatorname{supp}\varphi\subset B_{2r}(0)$. Using the method of [25], we obtain

$$ \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx=K_{1}+O \bigl(\varepsilon^{\frac {1}{2}}\bigr), \qquad \int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{6}\,dx=K_{2}+O \bigl(\varepsilon ^{\frac{3}{2}}\bigr), $$

(3.13)

and

$$ \int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{s}\,dx= \left \{ \textstyle\begin{array}{@{}l@{\quad}l} K\varepsilon^{\frac{s}{4}}, & s\in[2,3),\\ K\varepsilon^{\frac{3}{4}}|\ln\varepsilon|, & s=3,\\ K\varepsilon^{\frac{6-s}{4}}, & s\in(3,6), \end{array}\displaystyle \right . $$

(3.14)

where $K_{1}$, $K_{2}$, K are positive constants. Moreover, the best Sobolev constant $S=K_{1}K_{2}^{-\frac{1}{3}}$. By (3.13), we have

$$ \frac{\int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx}{(\int _{\mathrm{R}^{3}} w_{\varepsilon}^{6} \,dx)^{\frac{1}{3}}}=S+O\bigl(\varepsilon^{\frac{1}{2}}\bigr). $$

(3.15)

By Lemma 2.4, for this $w_{\varepsilon}$, there exists a unique $t(w_{\varepsilon})>0$ such that $t(w_{\varepsilon})w_{\varepsilon}\in {N}$. Thus $c_{1}< I(t(w_{\varepsilon})w_{\varepsilon})$. Using (2.1), for $t>0$, since $I(tw_{\varepsilon})\rightarrow-\infty$ as $t\rightarrow \infty$, we easily see that $I(tw_{\varepsilon})$ has a unique critical $t(w_{\varepsilon})>0$ which corresponds to its maximum, i.e. $I(t_{\varepsilon}w_{\varepsilon})=\max_{t>0}I(tw_{\varepsilon})$. It follows from (1) of Lemma 2.3, $I(tw_{\varepsilon})\rightarrow-\infty$ as $t\rightarrow\infty$ and the continuity of I that there exist two positive constants $t_{0}$ and $T_{0}$ such that $t_{0}< t_{\varepsilon}< T_{0}$. Let $I(t_{\varepsilon}w_{\varepsilon})=F(\varepsilon)+G(\varepsilon )+H(\varepsilon)$, where

$$\begin{aligned}& F(\varepsilon)=\frac{at_{\varepsilon}^{2}}{2} \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx + \frac{bt_{\varepsilon}^{4}}{4}\biggl( \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx \biggr)^{2} -\frac{ t_{\varepsilon}^{6}}{6} \int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx, \\& G(\varepsilon)=\frac{ t_{\varepsilon}^{6}}{6} \int_{\mathrm{R}^{3}}k(x_{0})| w_{\varepsilon}|^{6}\,dx- \frac{ t_{\varepsilon}^{6}}{6} \int_{\mathrm{R}^{3}}k(x)|w_{\varepsilon}|^{6}\,dx, \end{aligned}$$

and

$$H(\varepsilon)=\frac{t_{\varepsilon}^{2}}{2} \int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{2}\,dx-\frac{\mu t_{\varepsilon}^{2}}{2} \int_{\mathrm{R}^{3}}h(x)|w_{\varepsilon}|^{2}\,dx. $$

Set

$$\Phi(t)=\frac{at^{2}}{2} \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx+ \frac {bt^{4}}{4}\biggl( \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx \biggr)^{2}-\frac{ t^{6}}{6} \int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx. $$

Note that $\Phi(t)$ attains its maximum at

$$ t^{*}_{0}= \biggl(\frac{b(\int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx)^{2}+\sqrt {b^{2}(\int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx)^{4}+4a(\int_{\mathrm{R}^{3}}| \nabla w_{\varepsilon}|^{2}\,dx)^{2}\int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx}}{2\int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx} \biggr)^{\frac{1}{2}}, $$

then

$$ \max_{t\geq0}\Phi(t)=\Phi\bigl(t^{*}_{0}\bigr)= \frac{abS^{3}}{4\|k\|_{\infty}}+\frac{b^{3}S^{6}}{24\|k\|_{\infty}^{2}}+\frac{(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac {3}{2}}}{24\|k\|_{\infty}^{2}}+O\bigl( \varepsilon^{\frac{1}{2}}\bigr) $$

(3.16)

for $\varepsilon>0$ small enough. Then we have

$$ F(\varepsilon)\leq c^{*}+O\bigl(\varepsilon^{\frac{1}{2}}\bigr). $$

(3.17)

By (3.36) of [22], we have

$$ G(\varepsilon)\leq C\varepsilon^{\frac{1}{2}}. $$

(3.18)

From (3.38) of [22], (3.14), and the boundedness of $t_{\varepsilon}$, we obtain

$$\begin{aligned} H(\varepsilon)&=\frac{t_{\varepsilon}^{2}}{2} \int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{2}\,dx-\frac{\mu t_{\varepsilon}^{2}}{2} \int_{\mathrm{R}^{3}}h(x)|w_{\varepsilon}|^{2}\,dx \\ &\leq C\varepsilon^{\frac{1}{2}}-\mu C \varepsilon^{1-\frac{\beta}{2}}. \end{aligned}$$

(3.19)

Since $1<\beta<3$, for fixed $\mu>0$ we obtain

$$ \frac{H(\varepsilon)}{\varepsilon^{\frac{1}{2}}}\rightarrow-\infty, \quad\mbox{as }\varepsilon\rightarrow0. $$

(3.20)

It follows from (3.17), (3.18), and (3.20) that the proof of Lemma 3.3 is complete. □

Proof of Theorem 1.1

By the definition of $c_{1}$, there exists a sequence $\{u_{n}\}\subset N$ such that $I(u_{n})\rightarrow c_{1}$ as $n\rightarrow\infty$. Then we obtain

$$ \|u_{n}\|^{2}+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u_{n}|^{2}\,dx \biggr)^{2}- \int _{\mathrm{R}^{3}}\mu h(x)|u_{n}|^{2}\,dx= \int_{\mathrm{R}^{3}}k(x)|u_{n}|^{6}\,dx. $$

(3.21)

It follows from (3.21) and Lemma 2.2 that

$$\begin{aligned} c_{1}+o(1)&=\frac{1}{3}\biggl(\|u_{n} \|^{2}-\mu \int_{\mathrm{R}^{3}}h(x)|u_{n}|^{2}\,dx\biggr)+\biggl( \frac{b}{4}-\frac{b}{6}\biggr) \biggl( \int_{\mathrm{R}^{3}}|\nabla u_{n}|^{2}\,dx \biggr)^{2} \\ &\geq\frac{1}{3}\biggl(1-\frac{\mu}{\tilde{\mu}}\biggr)\|u_{n} \|^{2}, \end{aligned}$$

(3.22)

which implies the boundedness of $\{u_{n}\}$ in $H^{1}(\mathrm{R}^{3})$ since $0<\mu<\tilde{\mu}$. Then there exists a subsequence of $\{u_{n}\}$ still denoted by $\{u_{n}\}$ such that $u_{n}\rightharpoonup u$ in $H^{1}(\mathrm{R}^{3})$. By (2) of Lemma 3.2 and Lemma 3.3 we have $u\neq0$. By the definition of $t(u)$, we get $t(u)u\in{N}$. So $I(t(u)u)\geq c_{1}$. We claim that $u_{n}\rightarrow u$ in $H^{1}(\mathrm{R}^{3})$. Otherwise, by (1) and (3) of Lemma 3.2, we get $c_{1}>I(t(u)u)$ or $c_{1}>c^{*}$. In any case we get a contradiction since $c_{1}< c^{*}$. Therefore $\{u_{n}\}$ converges strongly to u. Thus $u\in{N}$ and $I(u)=c_{1}$. By the Lagrange multiplier rule, there exists $\theta\in R$ such that $I'(u)=\theta G'(u)$ and we have

$$0=\bigl\langle I'(u),u\bigr\rangle =\theta \biggl(2\|u \|^{2}+4b\biggl( \int _{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2}-6 \int_{\mathrm{R}^{3}} k(x)|u|^{6}\,dx-2\mu \int _{\mathrm{R}^{3}}h(x)|u|^{2}\,dx \biggr). $$

Since $u\in N$, we get

$$0=\theta \biggl(-4\biggl(\|u\|^{2}-\mu \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx\biggr) -2b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \biggr), $$

which implies $\theta=0$ and u is a nontrivial critical point of the functional I in $H^{1}(\mathrm{R}^{3})$. Therefore, the nonzero function u can solve equation (1.1), that is,

$$ -\biggl(a+b \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)\triangle u+u= k(x)|u|^{2^{*}-2}u+\mu h(x)u. $$

(3.23)

In (3.23), using $u^{-}=\max\{-u,0\}$ as a test function and integrating by parts, by (k₁), (h₂), and ($\mu_{1}$), we obtain

$$\begin{aligned} 0={}& \int_{\mathrm{R}^{3}}a\bigl|\nabla u^{-}\bigr|^{2}\,dx+ \int_{\mathrm{R}^{3}}\bigl|u^{-}\bigr|^{2}\,dx+b \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx \int_{\mathrm{R}^{3}}\bigl|\nabla u^{-}\bigr|^{2}\,dx \\ &{}+ \int_{\mathrm{R}^{3}}k(x)\bigl|u^{-}\bigr|^{2^{*}-2}\bigl|u^{-}\bigr|^{2}\,dx+ \int_{\mathrm{R}^{3}}\mu h(x)\bigl|u^{-}\bigr|^{2}\,dx\geq0, \end{aligned}$$

then $u^{-}=0$ and $u\geq0$. From Harnack’s inequality [27], we can infer that $u>0$ for all $x\in R^{3}$. Therefore, u is a positive solution of (1.1). The proof is complete by choosing $\omega_{0}=u$. □

4 Sign-changing solution

This subsection is devoted to proving the existence of sign-changing solution of equation (1.1). Let $\overline{{{N}}}=\{u=u^{+}-u^{-}\in H^{1}(\mathrm{R}^{3}):u^{+}\in{N}, u^{-}\in{N}\}$, where $u^{\pm}=\max\{ \pm u,0\}$. If $u^{+}\neq0$ and $u^{-}\neq0$, then u is called sign-changing function. We define $c_{2}=\inf_{u\in \overline{{{N}}}}I(u)$.

Lemma 4.1

Assume that ($\mu_{1}$), (k₁)-(k₂), and (h₁)-(h₂) hold, then for $\frac{3}{2}<\beta<3$, we have $c_{2}< c_{1}+c^{*}$.

Proof

By Lemma 2.4, first using the same argument as [22] or [28] we know that there is $s_{1}>0$ and $s_{2}\in R$ such that

$$ s_{1}\omega_{0}+s_{2}\omega_{\varepsilon}\in \overline{{N}}. $$

(4.1)

Next we prove that there exists $\varepsilon>0$ small enough such that

$$ \sup_{s_{1}>0,s_{2}\in\mathrm{R}}I(s_{1}\omega_{0}+s_{2} \omega _{\varepsilon})< c_{1}+c^{*}. $$

(4.2)

Obviously, it follows from (2) of Lemma 2.3 that for any $s_{1}>0$, $s_{2}\in R$ satisfying $\|s_{1}\psi_{1}+s_{2}\omega_{\varepsilon}\|>\rho$ that $I(s_{1}\omega_{0}+s_{2}\omega_{\varepsilon})<0$. We only estimate $I(s_{1}\omega _{0}+s_{2}\omega_{\varepsilon})$ for all $\|s_{1}\omega_{0}+s_{2}\omega_{\varepsilon}\|\leq\rho$. By calculation, we see

$$ I(s_{1}\omega_{0}+s_{2}\omega_{\varepsilon})=I(s_{1} \omega_{0})+\Pi _{1}+\Pi_{2}+\Pi_{3}+ \Pi_{4}+\Pi_{5}+\Pi_{6}, $$

(4.3)

where

$$\begin{aligned}& \Pi_{1}=\frac{as_{2}^{2}}{2} \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx+ \frac{bs_{2}^{4}}{4}\biggl( \int_{\mathrm{R}^{3}}|\nabla w_{\varepsilon}|^{2}\,dx \biggr)^{2}-\frac{ s_{2}^{6}}{6} \int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx, \\& \Pi_{2}=\frac{ s_{2}^{6}}{6} \int_{\mathrm{R}^{3}}k(x_{0})|w_{\varepsilon}|^{6}\,dx- \frac{ s_{2}^{6}}{6} \int_{\mathrm{R}^{3}}k(x)|w_{\varepsilon}|^{6}\,dx, \\& \Pi_{3}=\frac{1}{6} \int_{\mathrm{R}^{3}}k(x) \bigl(|s_{1}\omega _{0}|^{6}+|s_{2}w_{\varepsilon}|^{6}-|s_{1} \omega_{0}+s_{2}w_{\varepsilon}|^{6}\bigr)\,dx, \\& \Pi_{4}=\frac{s_{2}^{2}}{2} \int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{2}\,dx-\frac{\mu s_{2}^{2}}{2} \int_{\mathrm{R}^{3}}h(x)|w_{\varepsilon}|^{2}\,dx, \\& \begin{aligned}[b] \Pi_{5}={}&\frac{b}{4}\biggl[\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega _{0}+s_{2} \omega_{\varepsilon})\bigr|^{2}\,dx\biggr)^{2}-\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega _{0})\bigr|^{2}\,dx \biggr)^{2}\\ &{}-\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{2}\omega_{\varepsilon})\bigr|^{2}\,dx \biggr)^{2}\biggr], \end{aligned} \end{aligned}$$

and

$$\Pi_{6}= \int_{\mathrm{R}^{3}}\bigl(a\nabla(s_{1}\omega_{0}) \nabla( s_{2}\omega_{\varepsilon})+(s_{1} \omega_{0}) (s_{2}\omega_{\varepsilon})-\mu h(x) (s_{1}\omega_{0}) (s_{2}\omega_{\varepsilon}) \bigr)\,dx. $$

By (3.16), we obtain

$$ \sup_{s_{2}\in\mathrm{R}}\Pi_{1}=\frac{abS^{3}}{4\|k\|_{\infty}}+ \frac{b^{3}S^{6}}{24\|k\|_{\infty}^{2}}+\frac{(b^{2}S^{4}+4a\|k\|_{\infty}S)^{\frac {3}{2}}}{24\|k\|_{\infty}^{2}}+O\bigl(\varepsilon^{\frac{1}{2}}\bigr). $$

(4.4)

It follows from (3.18) that

$$ \Pi_{2}\leq C\varepsilon^{\frac{1}{2}}. $$

(4.5)

From the following elementary inequality:

$$|s+t|^{q}\geq|s|^{q}+|t|^{q}-C \bigl(|s|^{q-1}t+|t|^{q-1}s\bigr) \quad\mbox{for any } q\geq1 $$

and the fact of $\omega_{0}\in H^{1}(\mathrm{R}^{3}) \cap L^{\infty}(\mathrm{R}^{3})$ and (3.14) we have

$$\begin{aligned} \Pi_{3}&\leq C \int_{\mathrm{R}^{3}}k(x) \bigl(|\omega_{0}|^{5}\omega _{\varepsilon}+\omega_{0}|w_{\varepsilon}|^{5}\bigr)\,dx \\ &\leq\|k\|_{\infty}\|\omega_{0}\|_{\infty}\int_{\mathrm{R}^{3}}|w_{\varepsilon}|^{5}\,dx+\|k \|_{\infty}\|\omega_{0}^{5}\|_{\infty}\int_{\mathrm{R}^{3}} w_{\varepsilon}\,dx \\ &\leq C\varepsilon^{\frac{1}{4}}. \end{aligned}$$

(4.6)

By (3.19)

$$ \Pi_{4}\leq C\varepsilon^{\frac{1}{2}}-C\varepsilon^{1-\frac {\beta}{2}}. $$

(4.7)

And using (3.13), we have

$$\begin{aligned} \Pi_{5}\leq{}&\frac{b}{4}\biggl[4\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega _{0})\bigr|^{2}\,dx \biggr)^{2}+4\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{2}\omega_{\varepsilon})\bigr|^{2}\,dx \biggr)^{2} \\ &{}-\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega_{0})\bigr|^{2}\,dx \biggr)^{2}-\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{2}\omega_{\varepsilon})\bigr|^{2}\,dx \biggr)^{2}\biggr] \\ ={}&\frac{3b}{4}\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega_{0})\bigr|^{2}\,dx \biggr)^{2}+\frac {3b}{4}\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{2}\omega_{\varepsilon})\bigr|^{2}\,dx \biggr)^{2} \\ \leq{}& C+C\varepsilon^{\frac{1}{2}}. \end{aligned}$$

(4.8)

Since $\omega_{0}$ is a positive solution of (1.1), by the Sobolev inequality we obtain

$$\begin{aligned} \Pi_{6}&=s_{1}s_{2} \int_{\mathrm{R}^{3}}k(x)|\omega_{0}|^{5}\omega _{\varepsilon}\,dx-b \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega_{0})\bigr|^{2}\,dx \int_{\mathrm{R}^{3}}\nabla(s_{1}\omega_{0}) \nabla(s_{2}\omega_{\varepsilon})\,dx \\ &\leq\|k\|_{\infty}\|\omega_{0}^{5}\|_{\infty}\int_{\mathrm{R}^{3}} w_{\varepsilon}\,dx+b\biggl( \int_{\mathrm{R}^{3}}\bigl|\nabla(s_{1}\omega_{0})\bigr|^{2}\,dx \biggr)^{\frac{3}{2}}\biggl( \int _{\mathrm{R}^{3}}\bigl|\nabla(s_{2}\omega_{\varepsilon}) \bigr|^{2}\,dx\biggr)^{\frac{1}{2}} \\ &\leq C\varepsilon^{\frac{1}{4}}. \end{aligned}$$

(4.9)

It follows from (4.3)-(4.9) that for $\frac{3}{2}<\beta<3$ that

$$\begin{aligned} I(s_{1}\omega_{0}+s_{2}\omega_{\varepsilon})& \leq I(s_{1}\omega _{0})+c^{*}+C+C\varepsilon^{\frac{1}{4}}+C \varepsilon^{\frac {1}{2}}-C\varepsilon^{1-\frac{\beta}{2}} \\ &< I(s_{1}\omega_{0})+c^{*}=c_{1}+c^{*} \end{aligned}$$

as $\varepsilon\rightarrow0$, which implies that (4.2) holds. This finishes Lemma 4.1. □

Lemma 4.2

Suppose ($\mu_{1}$), (k₁)-(k₂), and (h₁)-(h₂), then for $\frac{3}{2}<\beta<3$, there exists $\omega _{1}\in\overline{{N}}$ such that $I(\omega_{1})=c_{2}$.

Proof

Let $\{u_{n}\}\subset\overline{{N}}$ be such that $I(u_{n})\rightarrow c_{2}$. Since $u_{n}\in\overline{{N}}$, we may assume that there exist constants $d_{1}$ and $d_{2}$ such that $I(u^{+}_{n})\rightarrow d_{1}$ and $I(u^{-}_{n})\rightarrow d_{2}$ and $d_{1}+d_{2}=c_{2}$. Then

$$ d_{1}\geq c_{1},\qquad d_{2}\geq c_{1}. $$

(4.10)

Just as the proof (3.22), we can prove the boundedness of $\{u^{+}_{n}\}$ and $\{u^{-}_{n}\}$. Going if necessary to a subsequence, we may assume that $u_{n}^{\pm}\rightharpoonup u^{\pm}$ in $H^{1}(\mathrm{R}^{3})$ as $n\rightarrow\infty$.

We claim $u^{+}\neq0$ and $u^{-}\neq0$. Arguing by contradiction, if $u^{+}=0$ or $u^{-}=0$, then by (4.10) and Lemma 3.2,

$$c_{1}+c^{*}\leq d_{2}+d_{1}=c_{2}, $$

which contradicts Lemma 4.1. Hence $u^{+}\neq0$ and $u^{-}\neq0$. We claim that $u^{\pm}_{n}\rightarrow u^{\pm}$ strongly in $H^{1}(\mathrm{R}^{3})$. Indeed, according to Lemma 3.2, we get one of the following:

• (i) $\{u^{+}_{n}\}$ converges strongly to $u^{+}$;

• (ii) $d_{1}>I(t(u^{+})u^{+})$;

• (iii) $d_{1}> c^{*}$;

and we also have one of the following:

• (iv) $\{u^{-}_{n}\}$ converges strongly to $u^{-}$;

• (v) $d_{2}>I(t(u^{-})u^{-})$;

• (vi) $d_{2}> c^{*}$.

We will prove that only cases (i) and (iv) hold. For example, in the case (i) and (v) or (ii) and (v), from $u^{+}-t(u^{-})u^{-}\in\overline{ {N}}$ or $t(u^{+})u^{+}-t(u^{-})u^{-}\in\overline{{N}}$, we have

$$c_{2}\leq I\bigl(u^{+}-t\bigl(u^{-}\bigr)u^{-}\bigr)=I\bigl(u^{+}\bigr)+I \bigl(-t\bigl(u^{-}\bigr)u^{-}\bigr)< d_{1}+d_{2}=c_{2}, $$

or

$$c_{2}\leq I\bigl(t\bigl(u^{+}\bigr)u^{+}-t\bigl(u^{-}\bigr)u^{-}\bigr)=I \bigl(t\bigl(u^{+}\bigr)u^{+}\bigr)+I\bigl(-t\bigl(u^{-}\bigr)u^{-}\bigr)< d_{1}+d_{2}=c_{2}. $$

Any one of the two inequalities is impossible. In the case (i) and (vi) or (ii) and (vi) or (iii) and (vi), we have

$$\begin{aligned}& c_{1}+c^{*}\leq I\bigl(u^{+}\bigr)+c^{*}< d_{1}+d_{2}=c_{2}, \\& c_{1}+c^{*}\leq I\bigl(t\bigl(u^{+}\bigr)u^{+}\bigr)+c^{*}< d_{1}+d_{2}=c_{2}, \\& c_{1}+c^{*}\leq c^{*}+c^{*}< d_{1}+d_{2}=c_{2}, \end{aligned}$$

and any one of the above three inequalities is a contradiction. Therefore we prove that only (i) and (iv) hold. Hence we obtain $\{ u^{+}_{n}\}$ and $\{u^{-}_{n}\}$ converge strongly to $u^{+}$ and $u^{-}$, respectively and we obtain $u^{+}, u^{-}\in N$. Denote $\omega _{1}=u^{+}-u^{-}$, then $\omega_{1}\in\overline{N}$ and $I(\omega _{1})=d_{1}+d_{2}=c_{2}$. □

Proof of Theorem 1.2

Now we show that $\omega_{1}$ is a critical point of I in $H^{1}(\mathrm{R}^{3})$. Arguing by contradiction, assume $I'(\omega_{1})\neq0$. For any $u\in{N}$ we claim that $\| G'(u)\|_{H^{-1}}=\sup_{\|v\|=1}|\langle G'(u),v\rangle|\neq0$. In fact, by the definition of N and Lemma 2.2, for any $u\in {N}$, we have

$$\begin{aligned} &\bigl\langle G'(u),u\bigr\rangle \\ &\quad=2 \biggl(\|u\|^{2}-\mu \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \biggr)\\ &\qquad{}+2b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} -6 \int_{\mathrm{R}^{3}}k(x)|u|^{6}\,dx \\ &\quad=2\biggl(\|u\|^{2}-\mu \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \biggr)+2b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \\ &\qquad{}-6\biggl(\|u\|^{2}-\mu \int _{\mathrm{R}^{3}}h(x)|u|^{2}\,dx+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2}\biggr) \\ &\quad=-4 \biggl(\|u\|^{2}-\mu \int_{\mathrm{R}^{3}}h(x)|u|^{2}\,dx+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \biggr)+2b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \\ &\quad\leq-4\biggl[\biggl(1-\frac{\mu}{\tilde{\mu}}\biggr)\|u\|^{2}+b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2} \biggr]+2b\biggl( \int_{\mathrm{R}^{3}}|\nabla u|^{2}\,dx\biggr)^{2}< 0. \end{aligned}$$

Then we can define

$$\Phi(u)=I'(u)- \biggl\langle I'(u), \frac{G'(u)}{\|G'(u)\| } \biggr\rangle \frac{G'(u)}{\|G'(u)\|},\quad u\in{N}. $$

Choosing $\lambda\in(0, \min\{\|u^{+}\|,\|u^{-}\|\}/3)$ such that $\|\Phi(v)-\Phi(u)\|\leq\frac{1}{2}\|\Phi(\omega_{1})\|$ for any $v\in N$ with $\|v-\omega_{1}\|\leq2\lambda$. Let $\chi:N\rightarrow[0,1]$ be a Lipschitz mapping such that

$$\chi(v)=\left \{ \textstyle\begin{array}{@{}l@{\quad}l} 0, & v\in N \mbox{ with } \|v-\omega_{1}\|\geq2\lambda,\\ 1, & v\in N \mbox{ with } \|v-\omega_{1}\|\leq\lambda, \end{array}\displaystyle \right . $$

and for positive constant $s_{0}$, $\eta:[0,s_{0}]\times N\rightarrow N$ be the solution of the differential equation

$$\eta(0,v)=0, \qquad \frac{d\eta(s,v)}{ds}=-\chi\bigl(\eta(s,v)\bigr)\Phi\bigl(\eta (s,v)\bigr), \quad\mbox{for } (s,v)\in[0,s_{0}]\times N. $$

We set

$$\psi(\tau)=t\bigl((1-\tau)\omega_{1}^{+}+\tau\omega_{1}^{-} \bigr) \bigl((1-\tau)\omega _{1}^{+}+\tau\omega_{1}^{-}, \xi( \tau)=\eta\bigl(s_{0},\psi(\tau)\bigr)\bigr),\quad \mbox{for } 0\leq \tau\leq1. $$

We now give the proof of the fact that $I(\xi(\tau))< I(u)$ for some $\tau\in(0,1)$. Obviously, if $\tau\in(0,\frac{1}{2})\cup(\frac {1}{2},1)$, we have $I(\xi(\frac{1}{2}))< I(\psi(\frac{1}{2}))< I(\omega _{1})$ and $I(\xi(\tau))\leq I(\psi(\tau))< I(\omega_{1})$.

Since $t(\xi^{+}(\tau))-t(\xi^{-}(\tau))\rightarrow-\infty$ as $\tau \rightarrow0+0$ and $t(\xi^{+}(\tau))-t(\xi^{-}(\tau))\rightarrow+\infty$ as $\tau\rightarrow1-0$, there exists $\tau_{1}\in(0,1)$ such that $t(\xi ^{+}(\tau))=t(\xi^{-}(\tau))$. Thus $\xi(\tau_{1})\in\overline{{N}}$ and $I(\xi(\tau_{1}))< I(\omega_{1})$, which contradicts the definition of $c_{2}$. Hence we get $I'(\omega_{1})=0$ and $\omega_{1}$ is a sign-changing solution of the problem (1.1). The proof of Theorem 1.2 is complete. □