Theory and Modern Applications

# The stability of bifurcating steady states for a spatially heterogeneous cooperative system with cross-diffusion

## Abstract

This paper investigates the stability of bifurcating steady states of a spatially heterogeneous cooperative system with cross-diffusion. According to the spectral analysis and the principle of exchange of stability, we show that the bifurcating steady states are stable.

## 1 Introduction

In this paper, we consider the following Lotka-Volterra cooperative system with cross-diffusion in a spatially heterogeneous environment:

$$\textstyle\begin{cases} u_{t}=\Delta u+u(\lambda-u+b(x)v), & x\in\Omega, t>0, \\ v_{t}=\Delta[(1+k\rho(x)u)v]+ v(\mu-v+d(x)u), & x\in\Omega, t>0, \\ \partial_{\nu}u=\partial_{\nu}v=0, & x\in\partial\Omega, t>0, \\ u(\cdot, 0)=u_{0}\geq0, \qquad v(\cdot, 0)=v_{0}\geq0, &x\in\Omega. \end{cases}$$
(1.1)

Here, Ω is a bounded domain in $$\mathbb{R}^{n}$$ ($$n\geq1$$) with smooth boundary Ω. u and v are the cooperative species. λ and μ represent the birth or death rates of the species which are real constants. $$b(x)\geq0$$ and $$d(x)\geq0$$ are continuous functions in Ω̅ representing the inter-specific interactions. $$\rho(x)$$ is a smooth positive function in Ω̅ satisfying $$\partial_{\nu}\rho(x)\vert_{\partial\Omega}=0$$, ν is the outer unit normal vector on Ω. The nonlinear diffusion term $$\Delta[\rho(x)uv]=\nabla[\rho(x)u\nabla v+v\nabla(\rho(x)u)]$$ is referred to as the cross-diffusion term. ρ is a positive constant in the homogeneous case, which means a tendency for v to move to the low density region of u. $$\rho(x)$$ is referred to as the cross-diffusion pressure. The term models a tendency for v to diffuse to the low density region of $$\rho(x)u$$. Furthermore, the tendency depends on both the population pressure of u and the heterogeneity of the environments. System (1.1) is a Lotka-Volterra cooperative system which was proposed by Shigesada et al.  to model the spatial segregation of two species. The Lotka-Volterra system with cross-diffusion is also called S-K-T model. For the S-K-T model with homogeneous coefficients, when one of the cross-diffusion rates is large enough, the existence and stability of spiky steady states have been deeply investigated by many authors, see . For the quasilinear cross-diffusion of fraction type of a prey predator model with spatially homogeneous coefficients, the existence and stability were studied in [5, 6]. For studying the effects of heterogeneous environments, some interesting papers have appeared over years. Du et al.  investigated degenerate effects of intra-specific pressures in some diffusive Lotka-Volterra systems. Hutson et al.  studied spatial effects of birth rates in some diffusive competitive models. Wang et al.  investigated a Lotka-Volterra cooperative system with cross-diffusion in a spatially heterogeneous environment. Applying the bifurcation theory and the Lyapunov-Schmidt reduction, they obtained the global bifurcation branch of positive steady states. They also proved that the spatial segregation of $$\rho(x)$$ and $$b(x)$$ could cause the positive solution curve to form an unbounded fish-hook shaped curve with parameter λ. Li et al.  showed some criteria for the stability of positive stationary solutions obtained in  and the Hopf bifurcation in certain circumstances. In this paper, we study the stability of the bifurcating steady states obtained in . Here we list the local bifurcation result and some preliminary results in , which will be used in this paper.

The corresponding steady state problem of (1.1) is as follows:

$$\textstyle\begin{cases} \Delta u+u(\lambda-u+b(x)v)=0, & x\in\Omega,\\ \Delta[(1+k\rho(x)u)v]+ v(\mu-v+d(x)u)=0, & x\in\Omega,\\ \partial_{\nu}u=\partial_{\nu}v=0, & x\in\partial\Omega. \end{cases}$$
(1.2)

Let $$V=(1+k\rho(x)u)v$$, then (1.2) can be rewritten as

$$\textstyle\begin{cases} \Delta u+u(\lambda-u+\frac{Vb(x)}{1+k\rho(x)u})=0, & x\in\Omega,\\ \Delta V+ \frac{V}{1+k\rho(x)u}(\mu-\frac{V}{1+k\rho(x)u}+d(x)u)=0, & x\in \Omega,\\ \partial_{\nu}u=\partial_{\nu}V=0, & x\in\partial\Omega. \end{cases}$$
(1.3)

The set of positive solutions of (1.3) is defined as follows:

$$\Gamma=\bigl\{ \bigl(u(x,s),V(x,s),\lambda(s)\bigr):s>0\bigr\} .$$

Obviously, (1.2) and (1.3) have the same semitrivial solution sets:

$$\Gamma_{u}=\bigl\{ (\lambda,0,\lambda):\lambda>0\bigr\} ,\quad \mbox{where }\bigl(u(x,s),V(x,s),\lambda(s)\bigr)=(\lambda,0,\lambda)$$

and

$$\Gamma_{V}=\bigl\{ (0,\mu,\lambda):\lambda\in\mathbb{R},\mu>0\bigr\} , \quad\mbox{where }\bigl(u(x,s),V(x,s),\lambda(s)\bigr)=(0,\mu,\lambda).$$

The positive functions $$\psi_{*}$$ and $$\phi^{*}$$ are defined by the solutions to the linear elliptic systems such that

\begin{aligned}& -\Delta\psi_{*}-\frac{\mu+\lambda_{*}d(x)}{1+\lambda_{*}k\rho (x)}\psi_{*}=0,\quad x\in\Omega, \partial_{\nu}\psi_{*}=0, x\in \partial\Omega, \int_{\Omega}(\psi_{*})^{2}\,dx=1; \end{aligned}
(1.4)
\begin{aligned}& -\Delta\phi^{*}-\bigl(\lambda^{*}+\mu b(x)\bigr) \phi^{*}=0,\quad x\in\Omega, \partial_{\nu}\phi^{*}=0,x\in \partial\Omega, \int_{\Omega}\bigl(\phi ^{*}\bigr)^{2}\,dx=1. \end{aligned}
(1.5)

Two sets are introduced as follows:

\begin{aligned}& S_{u}:=\biggl\{ (\lambda,\mu)\in\mathbb{R}^{2}: \lambda_{1}\biggl(-\frac{\mu +\lambda d(x)}{1+\lambda k\rho(x)}\biggr)=0\biggr\} ; \end{aligned}
(1.6)
\begin{aligned}& S_{V}:=\bigl\{ (\lambda,\mu)\in\mathbb{R}^{2}: \lambda_{1}\bigl(-\lambda-\mu b(x)\bigr)=0 \mbox{ for } \mu\geq0\bigr\} . \end{aligned}
(1.7)

The Banach spaces are defined as follows:

$$X:=W_{\nu}^{2,p}(\Omega)\times W_{\nu}^{2,p}( \Omega), \qquad Y:=L^{p}(\Omega)\times L^{p}(\Omega)\quad (p>n),$$

where $$W_{\nu}^{2,p}(\Omega):=\{u\in W^{2,p}(\Omega):\partial_{\nu }u\vert_{\partial\Omega}=0\}$$.

In this paper, we define $$\Vert u \Vert _{\infty}=\max_{x\in\overline{\Omega }} \vert u(x) \vert$$.

### Lemma 1.1

(Lemma 2.1 in )

For fixed $$\mu<0$$, there exists a monotone decreasing function $$\lambda=\lambda_{*}(\mu)>0$$ so that

$$S_{u}=\bigl\{ (\lambda,\mu)\in\mathbb{R}^{2}:\lambda= \lambda_{*}(\mu) \textit{ for }\mu< 0\bigr\} ,$$

with $$\lambda_{*}(0)=0$$ and $$\lim_{\mu\rightarrow-\infty}\lambda _{*}(\mu)=+\infty$$.

While if $$\mu>0$$, then for any λ, $$\lambda_{1}(-\frac{\mu +\lambda d(x)}{1+\lambda k\rho(x)})<0$$.

Furthermore, for fixed $$\mu>0$$, there exists a monotone decreasing function $$\lambda=\lambda^{*}(\mu)=\lambda_{1}(-\mu b(x))\leq0$$ such that

$$S_{V}=\bigl\{ (\lambda,\mu)\in\mathbb{R}^{2}:\lambda= \lambda^{*}(\mu) \textit{ for }\mu>0\bigr\} ,$$

with $$\lambda^{*}(0)=0$$.

### Lemma 1.2

(Lemma 2.5 in )

Let $$c=\frac{ \Vert \rho \Vert _{\infty}}{\min_{\overline{\Omega}}\rho}$$. Suppose $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}$$, when λ satisfies

$$\lambda\leq-c\mu \Vert b \Vert _{\infty} \quad\textit{if }\mu\geq0;\qquad \lambda\leq-\frac{\mu}{ \Vert d \Vert _{\infty}}\quad \textit{if }\mu< 0,$$

then (1.3) does not have any positive solutions.

### Lemma 1.3

(Lemma 2.6 in )

For any fixed $$(\mu,k,\rho(x),b(x),d(x))$$, the following local bifurcation properties hold true:

1. (i)

If $$\mu<0$$, then a branch of positive solutions of (1.3) bifurcates from $$\Gamma_{u}$$ if and only if $$\lambda=\lambda_{*}>0$$. Let $$(u(s),V(s),\lambda(s))\in X\times \mathbb{R}$$ be all positive solutions of (1.3) near $$(\lambda_{*},0,\lambda_{*})$$, then $$(u(s),V(s),\lambda(s))\in X\times\mathbb{R}$$ have the following structure:

$$\bigl\{ \bigl(u(s)=\lambda_{*}+s\bigl( \phi_{*}+s\overline{u}(s)\bigr),V(s)=s\bigl(\psi _{*}+s \overline{V}(s)\bigr),\lambda(s)\bigr)\in X\times\mathbb{R}:0< s\leq \delta_{*}\bigr\}$$
(1.8)

for $$(\phi_{*},\psi_{*})\in X$$ and some small $$\delta_{*}>0$$. $$\psi _{*}$$ is defined by (1.4) and

$$\phi_{*}=(-\Delta+\lambda_{*})^{-1} \biggl[\frac{\lambda _{*}b(x)}{1+\lambda_{*}k\rho(x)}\psi_{*}\biggr]>0.$$
(1.9)

$$(\overline{u}(s),\overline{V}(s),\lambda(s))$$ is a smooth function with $$\lambda(0)=\lambda_{*}$$ and $$\int_{\Omega}\overline{V}\psi_{*}\,dx=0$$.

2. (ii)

For $$\mu>0$$, a branch of positive solutions of (1.3) bifurcates from $$\Gamma_{V}$$ if and only if $$\lambda=\lambda^{*}<0$$. Let $$(\widetilde{u}(s),\widetilde {V}(s),\lambda(s))\in X\times\mathbb{R}$$ be all positive solutions of (1.3) near $$(0,\mu,\lambda^{*})$$, then $$(\widetilde{u}(s),\widetilde {V}(s),\lambda(s))\in X\times\mathbb{R}$$ have the following structure:

$$\bigl\{ \bigl(\widetilde{u}(s)=s\bigl(\phi^{*}+s \widehat{u}(s)\bigr),\widetilde {V}(s)=\mu+s\bigl(\psi^{*}+s \widehat{V}(s)\bigr),\lambda(s)\bigr):0< s\leq\delta ^{*}\bigr\}$$
(1.10)

for $$(\phi^{*},\psi^{*})\in X$$ and some small $$\delta^{*}>0$$. $$\phi ^{*}$$ is defined by (1.5) and

$$\psi^{*}=(-\Delta+\mu)^{-1}\bigl[\mu\bigl(d(x)+ \mu k\rho(x)\bigr)\phi^{*}\bigr].$$
(1.11)

$$(\widehat{u}(s),\widehat{V}(s),\lambda(s))$$ is a smooth function satisfying $$\lambda(0)=\lambda^{*}$$ and $$\int_{\Omega}\widehat {u}\phi^{*}\,dx=0$$.

In this paper, we first determine the bifurcating direction of bifurcation solutions. Then, by applying the spectral analysis and the principle of exchange stability, we prove that the bifurcation solutions near the bifurcation points are locally asymptotically stable. The plan of this paper is as follows. In Section 2, we prove that the bifurcating solutions near $$(\lambda_{*},0,\lambda_{*})$$ are locally asymptotically stable. In Section 3, we prove the local stability of bifurcating steady states near $$(0,\mu,\lambda^{*})$$. In Section 4, we give some numerical simulation results in order to verify the local stability of the bifurcation solutions obtained in this paper. Our conclusion is drawn in the final section.

## 2 The stability of bifurcating steady states near $$(\lambda _{*},0,\lambda_{*})$$

For system (1.1), let $$V=(1+k\rho(x)u)v$$, then system (1.1) can be reduced to the following:

$$\textstyle\begin{cases} u_{t}=\Delta u+u(\lambda-u+b(x)\frac{V}{1+k\rho(x)u}), &(x, t)\in\Omega\times(0, +\infty),\\ (\frac{V}{1+k\rho(x)u})_{t}=\Delta V+\frac{V}{1+k\rho(x)u}(\mu-\frac{V}{1+k\rho(x)u} +d(x)u),& (x, t)\in\Omega\times(0, +\infty),\\ \partial_{\nu}u=\partial_{\nu}V=0, & x\in\partial\Omega, \\ u(\cdot, 0)=u_{0}\geq0, \qquad V(\cdot, 0)=(1+k\rho(x)u_{0})v_{0}\geq0, & x\in\Omega. \end{cases}$$
(2.1)

We linearize (2.1) at $$(u(s), V(s))$$ and investigate the following eigenvalue problem:

$$\textstyle\begin{cases} \Delta u+[\lambda-2u(s)+\frac{b(x)V(s)}{(1+k\rho (x)u(s))^{2}}]u+\frac{b(x)u(s)}{1+k\rho(x)u(s)}V =\sigma u, & x\in \Omega, \\ \Delta V+[\frac{d(x)V(s)-k\rho(x)\mu V(s)}{(1+k\rho (x)u(s))^{2}}+\frac{2k\rho(x)V^{2}(s)}{(1+k\rho(x)u(s))^{3}}]u+ [\frac{\mu+d(x)u(s)}{1+k\rho(x)u(s)}-\frac{2V(s)}{(1+k\rho(x)u(s))^{2}}]V\\ \quad =\frac{1}{1+k\rho(x)u(s)}\sigma V-\frac{k\rho(x)V(s)}{(1+k\rho(x)u(s))^{2}}\sigma u, & x\in \Omega, \\ \partial_{\nu}u=\partial_{\nu}V=0, & x\in\partial\Omega. \end{cases}$$
(2.2)

The operator can be defined as follows:

$$G:X\times\mathbb{R}\rightarrow Y$$ with

$\begin{array}{rl}G\left(u,V,\lambda \right)& =\left(\begin{array}{c}{G}_{1}\left(u,V,\lambda \right)\\ {G}_{2}\left(u,V,\lambda \right)\end{array}\right)\\ & =\left(\begin{array}{c}\mathrm{\Delta }u+u\left(\lambda -u+b\left(x\right)\frac{V}{1+k\rho \left(x\right)u}\right)\\ \mathrm{\Delta }V+\frac{V}{1+k\rho \left(x\right)u}\left(\mu -\frac{V}{1+k\rho \left(x\right)u}+d\left(x\right)u\right)\end{array}\right).\end{array}$
(2.3)

We can compute

${G}_{\left(u,V\right)}\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)=\left(\begin{array}{c}\mathrm{\Delta }u-{\lambda }_{\ast }u+\frac{{\lambda }_{\ast }b\left(x\right)}{1+{\lambda }_{\ast }k\rho \left(x\right)}V\\ \mathrm{\Delta }V+\frac{\mu +{\lambda }_{\ast }d\left(x\right)}{1+{\lambda }_{\ast }k\rho \left(x\right)}V\end{array}\right),$
(2.4)
\begin{aligned}& N\bigl(G_{(u,V)}(\lambda_{*},0, \lambda_{*})\bigr)=\operatorname{span}\{\phi_{*},\psi_{*}\}, \end{aligned}
(2.5)

here N denotes the null space.

(2.2) can be written as

$\begin{array}{rl}& {G}_{\left(u,V\right)}\left(u\left(s\right),V\left(s\right),\lambda \right)\left(\begin{array}{c}u\\ V\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=\left(\begin{array}{cc}1& 0\\ \frac{-k\rho \left(x\right)V\left(s\right)}{{\left(1+k\rho \left(x\right)u\left(s\right)\right)}^{2}}& \frac{1}{1+k\rho \left(x\right)u\left(s\right)}\end{array}\right)\left(\begin{array}{c}\sigma u\\ \sigma V\end{array}\right).\end{array}$
(2.6)

An operator can be introduced by

$$L:X\times\mathbb{R}\rightarrow Y$$

with

$\begin{array}{rl}& L\left(u\left(s\right),V\left(s\right),\lambda \right)\\ & \phantom{\rule{1em}{0ex}}={\left(\begin{array}{cc}1& 0\\ \frac{-k\rho \left(x\right)V\left(s\right)}{{\left(1+k\rho \left(x\right)u\left(s\right)\right)}^{2}}& \frac{1}{1+k\rho \left(x\right)u\left(s\right)}\end{array}\right)}^{-1}{G}_{\left(u,V\right)}\left(u\left(s\right),V\left(s\right),\lambda \right).\end{array}$
(2.7)

Taking (2.6) and (2.7) in consideration, (2.2) can be reduced as follows:

$L\left(u\left(s\right),V\left(s\right),\lambda \right)\left(\begin{array}{c}u\\ V\end{array}\right)=\left(\begin{array}{c}\sigma u\\ \sigma V\end{array}\right).$
(2.8)

In the following, first we will show the bifurcating direction. Then, by applying the bifurcating direction and the spectral analysis, we will prove the stability of positive solution bifurcating from $$(\lambda_{*}, 0, \lambda_{*})$$.

According to Shi  (Theorem 2.1 and (4.5)), we can define the functional

$$l_{1}:X\rightarrow\mathbb{R}\quad \mbox{by }\bigl\langle [f,g],l_{1}\bigr\rangle := \int _{\Omega}g\psi_{*}\,dx.$$
(2.9)

### Lemma 2.1

Let $$c=\frac{ \Vert \rho \Vert _{\infty}}{\min_{\overline{\Omega}}\rho}$$. For any fixed $$(\mu,k,\rho(x),b(x),d(x))$$ with $$\mu<0$$, $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}$$, when d and ρ are constant functions, the bifurcating direction satisfies

$$\frac{d\lambda(s)}{ds}\bigg\vert _{s=0}=-\frac{1}{2} \frac{\langle G_{(u,V)(u,V)}(\lambda_{*}, 0, \lambda_{*})[\bigl( {\scriptsize\begin{matrix}{} \phi_{*} \cr \psi_{*} \end{matrix}} \bigr), \bigl( {\scriptsize\begin{matrix}{} \phi_{*} \cr \psi_{*} \end{matrix}} \bigr) ], l_{1}\rangle}{ \langle G_{(u,V),\lambda}(\lambda_{*},0,\lambda_{*})\bigl( {\scriptsize\begin{matrix}{} \phi_{*} \cr \psi_{*} \end{matrix}} \bigr), l_{1}\rangle}>0.$$
(2.10)

### Proof

Applying the bifurcation formula in  (Theorem 2.1 and (4.5)),we have the above expression of $$\frac{d\lambda(s)}{ds}\vert _{s=0}$$. In the following, we will prove $$\frac{d\lambda(s)}{ds}\vert _{s=0}>0$$. It follows from (2.3) that

${\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{1}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{1}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{1}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{1}}{\partial {V}^{2}}\end{array}\right)|}_{\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)}=\left(\begin{array}{cc}-2& \frac{b\left(x\right)}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}\\ \frac{b\left(x\right)}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}& 0\end{array}\right).$
(2.11)

Thus

$\begin{array}{rl}& \left(\begin{array}{cc}{\varphi }_{\ast }& {\psi }_{\ast }\end{array}\right){\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{1}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{1}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{1}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{1}}{\partial {V}^{2}}\end{array}\right)|}_{\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)}\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=-2{\varphi }_{\ast }^{2}+\frac{2b\left(x\right){\varphi }_{\ast }{\psi }_{\ast }}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}.\end{array}$
(2.12)

Using a similar calculation, we have

$\begin{array}{rl}& \left(\begin{array}{cc}{\varphi }_{\ast }& {\psi }_{\ast }\end{array}\right){\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{2}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{2}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{2}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{2}}{\partial {V}^{2}}\end{array}\right)|}_{\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)}\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=\frac{2\left(d\left(x\right)-\mu k\rho \left(x\right)\right){\varphi }_{\ast }{\psi }_{\ast }-2{\left({\psi }_{\ast }\right)}^{2}}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}.\end{array}$
(2.13)

According to (2.12) and (2.13), we obtain

$\begin{array}{rl}& {G}_{\left(u,V\right)\left(u,V\right)}\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left[\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)\right]\\ & \phantom{\rule{1em}{0ex}}=\left(\begin{array}{c}-2{\varphi }_{\ast }^{2}+\frac{2b\left(x\right){\varphi }_{\ast }{\psi }_{\ast }}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}\\ \frac{2\left(d\left(x\right)-\mu k\rho \left(x\right)\right){\varphi }_{\ast }{\psi }_{\ast }-2{\left({\psi }_{\ast }\right)}^{2}}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}\end{array}\right).\end{array}$
(2.14)

From (2.14), we can calculate the following equation:

$\begin{array}{rl}& 〈{G}_{\left(u,V\right)\left(u,V\right)}\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left[\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)\right],{l}_{1}〉\\ & \phantom{\rule{1em}{0ex}}=2{\int }_{\mathrm{\Omega }}\frac{\left[\left(d\left(x\right)-\mu k\rho \left(x\right)\right){\varphi }_{\ast }-{\psi }_{\ast }\right]{\left({\psi }_{\ast }\right)}^{2}}{{\left(1+k\rho \left(x\right){\lambda }_{\ast }\right)}^{2}}\phantom{\rule{0.2em}{0ex}}dx.\end{array}$
(2.15)

Combining (1.4), (2.4) and (2.5) gives the following equation:

\begin{aligned}[b] \bigl(d(x)-\mu k\rho(x)\bigr) \phi_{*}-\psi_{*}&=\frac{d(x)-\mu k\rho (x)}{\lambda_{*}}\Delta\phi_{*} \\ &\quad{}+\frac{b(x)d(x)-\mu k\rho(x) b(x)-1-\lambda_{*}k\rho (x)}{1+\lambda_{*}k\rho(x)}\psi_{*}. \end{aligned}
(2.16)

When d and ρ are constant functions, we have $$\psi_{*}=\frac {1}{ \vert \Omega \vert ^{\frac{1}{2}}}$$ by (1.4).

It follows from (2.9), (2.15), (2.16) and Lemma 1.2 that

$\begin{array}{rl}& 〈{G}_{\left(u,V\right)\left(u,V\right)}\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left[\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)\right],{l}_{1}〉\\ & \phantom{\rule{1em}{0ex}}=2{\int }_{\mathrm{\Omega }}\frac{\left[b\left(x\right)d-\mu k\rho b\left(x\right)-1-{\lambda }_{\ast }k\rho \right]{\left({\psi }_{\ast }\right)}^{3}}{{\left(1+k\rho {\lambda }_{\ast }\right)}^{3}}\phantom{\rule{0.2em}{0ex}}dx<0.\end{array}$
(2.17)

In fact, according to Lemma 1.2, we obtain $$\lambda_{*} \Vert d \Vert _{\infty}>-\mu$$ and $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}<1$$. These imply $$\lambda_{*} \Vert d \Vert _{\infty}b>-\mu b$$ and $$\lambda_{*}>-\mu b$$. Thus we have (2.17).

After a simple calculation, we have

(2.18)

From (2.17) and (2.18), we complete the proof of Lemma 2.1. □

Next, we will investigate the stability of the bifurcating steady state $$(u(s),V(s))$$ defined by (1.8).

### Theorem 2.2

Let $$c=\frac{ \Vert \rho \Vert _{\infty}}{\min_{\overline{\Omega}}\rho}$$. For any fixed $$(\mu,k,\rho(x),b(x),d(x))$$ with $$\mu<0$$, $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}$$, when d and ρ are constant functions, the bifurcating solution $$(u(s), V(s))$$ defined by (1.8) of system (2.1) is locally asymptotically stable.

### Proof

In the following, first we need to show that 0 is the first eigenvalue of $$L(\lambda_{*},0,\lambda_{*})$$. For any fixed $$(u,V)\in X$$, we have

$\begin{array}{rl}L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)& ={\left(\begin{array}{cc}1& 0\\ 0& \frac{1}{1+{\lambda }_{\ast }k\rho }\end{array}\right)}^{-1}{G}_{\left(u,V\right)}\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)\\ & =\left(\begin{array}{cc}1& 0\\ 0& 1+{\lambda }_{\ast }k\rho \end{array}\right)\left(\begin{array}{c}\mathrm{\Delta }u-{\lambda }_{\ast }u+\frac{{\lambda }_{\ast }b\left(x\right)}{1+{\lambda }_{\ast }k\rho }V\\ \mathrm{\Delta }V+\frac{\mu +{\lambda }_{\ast }d}{1+{\lambda }_{\ast }k\rho }V\end{array}\right)\\ & =\left(\begin{array}{c}\mathrm{\Delta }u-{\lambda }_{\ast }u+\frac{{\lambda }_{\ast }b\left(x\right)}{1+{\lambda }_{\ast }k\rho }V\\ \left(1+{\lambda }_{\ast }k\rho \right)\left(\mathrm{\Delta }V+\frac{\mu +{\lambda }_{\ast }d}{1+{\lambda }_{\ast }k\rho }V\right)\end{array}\right)=0.\end{array}$
(2.19)

It follows from (1.4), (2.4) and (2.5) that

$L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$
(2.20)

Then 0 is an eigenvalue of the operator $$L(\lambda_{*},0,\lambda _{*})$$. Next, we will show that 0 is the first eigenvalue of the operator $$L(\lambda_{*},0,\lambda_{*})$$. Otherwise, there is a positive eigenvalue $$\sigma_{1}$$ of $$L(\lambda_{*},0,\lambda_{*})$$ with the corresponding eigenfunction $$\bigl({\scriptsize\begin{matrix}{}u_{1} \cr V_{1}\end{matrix}} \bigr)\in X$$ satisfying

$L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{u}_{1}\\ {V}_{1}\end{array}\right)=\left(\begin{array}{c}{\sigma }_{1}{u}_{1}\\ {\sigma }_{1}{V}_{1}\end{array}\right),$
(2.21)

that is,

\begin{aligned} \textstyle\begin{cases} \Delta u_{1}-\lambda_{*}u_{1}+\frac{\lambda_{*}b(x)}{1+\lambda _{*}k\rho}V_{1}=\sigma_{1}u_{1}, &x\in\Omega, \\ (1+\lambda_{*}k\rho)(\Delta V_{1}+\frac{\mu+\lambda _{*}d}{1+\lambda_{*}k\rho}V_{1})=\sigma_{1}V_{1}, &x\in\Omega, \\ \partial_{\nu}u_{1}=\partial_{\nu}V_{1}=0, &x\in\partial\Omega. \end{cases}\displaystyle \end{aligned}
(2.22)

If $$V_{1}=0$$ and $$u_{1}\neq0$$ hold, (2.22) yields

\begin{aligned}& \Delta u_{1}-\lambda_{*}u_{1}= \sigma_{1}u_{1}\quad (\sigma_{1}>0), \end{aligned}
(2.23)
\begin{aligned}& u_{1}=(-\Delta+\lambda_{*})^{-1}(- \sigma_{1}u_{1}), \end{aligned}
(2.24)

which is impossible since $$(-\Delta+\lambda_{*})^{-1}(-\sigma _{1}u_{1})$$ is negative if $$-\sigma_{1}u_{1}$$ is negative, thus $$V_{1}\neq0$$. The second equation of (2.22) can be reduced as follows:

$$\Delta V_{1}+\frac{\mu+\lambda_{*}d}{1+\lambda_{*}k\rho }V_{1}= \frac{\sigma_{1}}{1+\lambda_{*}k\rho}V_{1}.$$
(2.25)

By (1.4), (2.25) and the scalar elliptic equation theorem, 0 is the first eigenvalue of (2.25), which contradicts $$\sigma_{1}>0$$. Then we have proved that 0 is the first eigenvalue of $$L(\lambda_{*},0,\lambda_{*})$$ and the other eigenvalues are negative.

In virtue of Proposition I.7.2 in , for small $$0< s<\delta$$, there are a perturbed eigenvalue $$\sigma(s)$$ and continuous differential functions $$\omega_{1}(s), \omega_{2}(s) \in X\cap \operatorname{Range}(G_{(u, V)}(\lambda_{*},0,\lambda_{*}))$$ satisfying

$L\left(u\left(s\right),V\left(s\right),\lambda \left(s\right)\right)\left(\begin{array}{c}{\varphi }_{\ast }+{\omega }_{1}\left(s\right)\\ {\psi }_{\ast }+{\omega }_{2}\left(s\right)\end{array}\right)=\sigma \left(s\right)\left(\begin{array}{c}{\varphi }_{\ast }+{\omega }_{1}\left(s\right)\\ {\psi }_{\ast }+{\omega }_{2}\left(s\right)\end{array}\right),$
(2.26)

with $$\sigma(0)=0$$, $$\omega_{1}(0)=0$$, $$\omega_{2}(0)=0$$.

Similarly, there exist a perturbed eigenvalue $$\sigma(\lambda)$$ and continuous differential functions $$\omega_{1}(\lambda), \omega_{2}(\lambda) \in X\cap \operatorname{Range}(G_{(u, V)}(\lambda_{*},0,\lambda_{*}))$$ satisfying

$L\left(\lambda ,0,\lambda \right)\left(\begin{array}{c}{\varphi }_{\ast }+{\omega }_{1}\left(\lambda \right)\\ {\psi }_{\ast }+{\omega }_{2}\left(\lambda \right)\end{array}\right)=\sigma \left(\lambda \right)\left(\begin{array}{c}{\varphi }_{\ast }+{\omega }_{1}\left(\lambda \right)\\ {\psi }_{\ast }+{\omega }_{2}\left(\lambda \right)\end{array}\right),$
(2.27)

with $$\sigma(\lambda_{*})=\omega_{1}(\lambda_{*})=\omega _{2}(\lambda_{*})=0$$.

Differentiating equation (2.27) with respect to λ at $$\lambda_{*}$$ yields

$\frac{d}{d\lambda }L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)+L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{\omega }_{1}^{\prime }\left({\lambda }_{\ast }\right)\\ {\omega }_{2}^{\prime }\left({\lambda }_{\ast }\right)\end{array}\right)={\sigma }^{\prime }\left({\lambda }_{\ast }\right)\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right),$
(2.28)

where $$\sigma^{\prime}(\lambda)=\frac{d}{d\lambda}\sigma(\lambda)$$.

It follows from (1.4), (2.9) and (2.19) that

$〈\frac{d}{d\lambda }L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right),{l}_{1}〉={\sigma }^{\prime }\left({\lambda }_{\ast }\right).$
(2.29)

From (2.19), we can calculate the following equation:

$\frac{d}{d\lambda }L\left({\lambda }_{\ast },0,{\lambda }_{\ast }\right)\left(\begin{array}{c}{\varphi }_{\ast }\\ {\psi }_{\ast }\end{array}\right)=\left(\begin{array}{c}-{\varphi }_{\ast }+\frac{b\left(x\right){\psi }_{\ast }}{{\left(1+{\lambda }_{\ast }k\rho \right)}^{2}}\\ \frac{\left(d-\mu k\rho \right){\psi }_{\ast }}{1+{\lambda }_{\ast }k\rho }\end{array}\right).$
(2.30)

Combining (2.29) and (2.30), we obtain

$$\sigma'(\lambda_{*})= \int_{\Omega}\frac{(d-\mu k\rho)\psi _{*}^{2}}{1+\lambda_{*}k\rho}\,dx>0.$$
(2.31)

Here, $$\sigma'(\lambda_{*})>0$$ yields that $$(\lambda_{*},0,\lambda_{*})$$ is stable for $$\lambda<\lambda_{*}$$ and unstable for $$\lambda>\lambda_{*}$$.

Using formula I.7.40 in , we obtain

$$-\dot{\sigma}(0)=\dot{\lambda}(0)\sigma'( \lambda_{*}),$$
(2.32)

where $$\dot{\sigma}(s)=\frac{d}{ds}\sigma(s)$$.

It follows from Lemma 2.1 and (2.32) that

$$\dot{\sigma}(0)< 0,$$
(2.33)

which yields $$\sigma(s)<0$$ for small $$s>0$$, so the bifurcating solution $$(u(s),V(s))$$ defined by (1.8) of system (2.1) is locally asymptotically stable. □

## 3 The stability of bifurcating steady states near $$(0,\mu ,\lambda^{*})$$

In this section, we use a method similar to that in Section 2 in order to investigate the stability of positive solutions bifurcating from $$(0,\mu,\lambda^{*})$$.

According to Shi  (Theorem 2.1 and (4.5)), we can define the functional

$$l_{2}:X\rightarrow\mathbb{R}\quad\mbox{by }\bigl\langle [f,g],l_{2}\bigr\rangle := \int_{\Omega}f\phi^{*}\,dx.$$
(3.1)

### Lemma 3.1

Let $$c=\frac{ \Vert \rho \Vert _{\infty}}{\min_{\overline{\Omega}}\rho}$$. For any fixed $$(\mu,k,\rho(x),b(x),d(x))$$ with $$\mu>0$$, $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}$$, when $$b(x)$$ is a constant function, the bifurcating direction satisfies

$$\frac{d\lambda(s)}{ds}\bigg\vert _{s=0}=-\frac{1}{2} \frac{\langle G_{(u,V)(u,V)}(0,\mu, \lambda^{*})[\bigl( {\scriptsize\begin{matrix}{} \phi^{*} \cr \psi^{*} \end{matrix}} \bigr), \bigl( {\scriptsize\begin{matrix}{} \phi^{*} \cr \psi^{*} \end{matrix}} \bigr) ], l_{2}\rangle}{ \langle G_{(u,V),\lambda}(0,\mu,\lambda^{*})\bigl( {\scriptsize\begin{matrix}{} \phi^{*} \cr \psi^{*} \end{matrix}} \bigr), l_{2}\rangle}>0.$$
(3.2)

### Proof

Using the bifurcation formula in  (Theorem 2.1 and (4.5)), we obtain the above expression of $$\frac{d\lambda(s)}{ds}\vert _{s=0}$$. According to (2.3), we have

${\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{1}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{1}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{1}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{1}}{\partial {V}^{2}}\end{array}\right)|}_{\left(0,\mu ,{\lambda }^{\ast }\right)}=\left(\begin{array}{cc}-2-2b\left(x\right)\mu k\rho \left(x\right)& b\left(x\right)\\ b\left(x\right)& 0\end{array}\right).$
(3.3)

Thus

$\begin{array}{rl}& \left(\begin{array}{cc}{\varphi }^{\ast }& {\psi }^{\ast }\end{array}\right){\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{1}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{1}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{1}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{1}}{\partial {V}^{2}}\end{array}\right)|}_{\left(0,\mu ,{\lambda }^{\ast }\right)}\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=-2{\left({\varphi }^{\ast }\right)}^{2}-2b\left(x\right)\mu k\rho \left(x\right){\left({\varphi }^{\ast }\right)}^{2}+2b\left(x\right){\varphi }^{\ast }{\psi }^{\ast }.\end{array}$
(3.4)

After a simple calculation, we obtain

$\begin{array}{rl}& \left(\begin{array}{cc}{\varphi }^{\ast }& {\psi }^{\ast }\end{array}\right){\left(\begin{array}{cc}\frac{{\partial }^{2}{G}_{2}}{\partial {u}^{2}}& \frac{{\partial }^{2}{G}_{2}}{\partial u\phantom{\rule{0.2em}{0ex}}\partial V}\\ \frac{{\partial }^{2}{G}_{2}}{\partial V\phantom{\rule{0.2em}{0ex}}\partial u}& \frac{{\partial }^{2}{G}_{2}}{\partial {V}^{2}}\end{array}\right)|}_{\left(0,\mu ,{\lambda }^{\ast }\right)}\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=-4{\mu }^{2}{k}^{2}{\rho }^{2}\left(x\right){\left({\varphi }^{\ast }\right)}^{2}-2\mu k\rho \left(x\right)d\left(x\right){\left({\varphi }^{\ast }\right)}^{2}+6\mu k\rho \left(x\right){\varphi }^{\ast }{\psi }^{\ast }\\ & \phantom{\rule{2em}{0ex}}+2d\left(x\right){\varphi }^{\ast }{\psi }^{\ast }-2{\left({\psi }^{\ast }\right)}^{2}.\end{array}$
(3.5)

It follows from (3.4) and (3.5) that

$\begin{array}{rl}& {G}_{\left(u,V\right)\left(u,V\right)}\left(0,\mu ,{\lambda }^{\ast }\right)\left[\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)\right]\\ & \phantom{\rule{1em}{0ex}}=\left(\begin{array}{c}-2{\left({\varphi }^{\ast }\right)}^{2}-2b\left(x\right)\mu k\rho \left(x\right){\left({\varphi }^{\ast }\right)}^{2}+2b\left(x\right){\varphi }^{\ast }{\psi }^{\ast }\\ -4{\mu }^{2}{k}^{2}{\rho }^{2}\left(x\right){\left({\varphi }^{\ast }\right)}^{2}-2\mu k\rho \left(x\right)d\left(x\right){\left({\varphi }^{\ast }\right)}^{2}+6\mu k\rho \left(x\right){\varphi }^{\ast }{\psi }^{\ast }\\ +2d\left(x\right){\varphi }^{\ast }{\psi }^{\ast }-2{\left({\psi }^{\ast }\right)}^{2}\end{array}\right).\end{array}$
(3.6)

Combining (3.1) and (3.6) give the following equation:

$\begin{array}{rl}& 〈{G}_{\left(u,V\right)\left(u,V\right)}\left(0,\mu ,{\lambda }^{\ast }\right)\left[\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)\right],{l}_{2}〉\\ & \phantom{\rule{1em}{0ex}}={\int }_{\mathrm{\Omega }}\left(-2{\left({\varphi }^{\ast }\right)}^{2}-2b\left(x\right)\mu k\rho \left(x\right){\left({\varphi }^{\ast }\right)}^{2}+2b\left(x\right){\varphi }^{\ast }{\psi }^{\ast }\right){\varphi }^{\ast }\phantom{\rule{0.2em}{0ex}}dx.\end{array}$
(3.7)

Using (1.5) and (1.11), we obtain

$$-2\phi^{*}-2b\mu k\rho(x)\phi^{*}+2b \psi^{*}=\frac{2b}{\mu}\Delta \psi_{*}+\bigl(2bd(x)-2 \bigr)\phi^{*}.$$
(3.8)

When $$b(x)$$ is a constant function, we have $$\phi^{*}=\frac {1}{ \vert \Omega \vert ^{\frac{1}{2}}}$$ by (1.5).

It follows from (3.7), (3.8) and $$\Vert b \Vert _{\infty}\Vert \,d\Vert _{\infty}<\frac{1}{c}<1$$ that

$\begin{array}{rl}& 〈{G}_{\left(u,V\right)\left(u,V\right)}\left(0,\mu ,{\lambda }^{\ast }\right)\left[\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)\right],{l}_{2}〉\\ & \phantom{\rule{1em}{0ex}}={\int }_{\mathrm{\Omega }}\left(2bd\left(x\right)-2\right){\left({\varphi }^{\ast }\right)}^{2}\phantom{\rule{0.2em}{0ex}}dx<0.\end{array}$
(3.9)

It is easy to compute that

$\begin{array}{rl}& 〈{G}_{\left(u,V\right),\lambda }\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),{l}_{2}〉\\ & \phantom{\rule{1em}{0ex}}={\int }_{\mathrm{\Omega }}{\left({\varphi }^{\ast }\right)}^{2}\phantom{\rule{0.2em}{0ex}}dx>0.\end{array}$
(3.10)

From (3.9) and (3.10), we complete the proof of Lemma 3.1. □

We linearize (2.1) at $$(\widetilde{u}(s), \widetilde{V}(s))$$ and study the following eigenvalue problem:

$$\textstyle\begin{cases} \Delta u+[\lambda-2u+\frac{b\widetilde{V}(s)}{(1+k\rho(x)\widetilde {u}(s))^{2}}]u+\frac{b\widetilde{u}(s)}{1+k\rho(x)\widetilde{u}(s)}V =\widetilde{\sigma}u, & x\in \Omega, \\ \Delta V+[\frac{d(x)\widetilde{V}(s)-k\rho(x)\mu\widetilde {V}(s)}{(1+k\rho(x)\widetilde{u}(s))^{2}}+\frac{2k\rho(x)\widetilde {V}^{2}(s)}{(1+k\rho(x)\widetilde{u}(s))^{3}}]u\\ \quad{}+ [\frac{\mu+d(x)\widetilde{u}(s)}{1+k\rho(x)\widetilde{u}(s)}- \frac{2\widetilde{V}(s)}{(1+k\rho(x)\widetilde{u}(s))^{2}}]V =\frac {1}{1+k\rho(x)\widetilde{u}(s)}\widetilde{\sigma} V-\frac{k\rho (x)\widetilde{V}(s)}{(1+k\rho(x)\widetilde{u}(s))^{2}}\widetilde {\sigma} u,&x\in\Omega, \\ \partial_{\nu}u=\partial_{\nu}V=0, & x\in\partial\Omega. \end{cases}$$
(3.11)

By (2.3), we can compute

${G}_{\left(u,V\right)}\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)=\left(\begin{array}{c}\mathrm{\Delta }u+\left({\lambda }^{\ast }+\mu b\right)u\\ \mathrm{\Delta }V+\mu \left(d\left(x\right)+\mu k\rho \left(x\right)\right)u-\mu V\end{array}\right),$
(3.12)
\begin{aligned}& N\bigl(G_{(u,V)}\bigl(0,\mu,\lambda^{*}\bigr) \bigr)=\operatorname{span}\bigl\{ \phi^{*},\psi^{*}\bigr\} , \end{aligned}
(3.13)

where N denotes the null space.

System (3.11) can be converted to

${G}_{\left(u,V\right)}\left(\stackrel{˜}{u}\left(s\right),\stackrel{˜}{V}\left(s\right),\lambda \right)\left(\begin{array}{c}u\\ V\end{array}\right)=\left(\begin{array}{cc}1& 0\\ \frac{-k\rho \left(x\right)\stackrel{˜}{V}\left(s\right)}{{\left(1+k\rho \left(x\right)\stackrel{˜}{u}\left(s\right)\right)}^{2}}& \frac{1}{1+k\rho \left(x\right)\stackrel{˜}{u}\left(s\right)}\end{array}\right)\left(\begin{array}{c}\stackrel{˜}{\sigma }u\\ \stackrel{˜}{\sigma }V\end{array}\right).$
(3.14)

Taking (2.7) and (3.14) into account, (3.11) can be reduced as

$L\left(\stackrel{˜}{u}\left(s\right),\stackrel{˜}{V}\left(s\right),\lambda \right)\left(\begin{array}{c}u\\ V\end{array}\right)=\left(\begin{array}{c}\stackrel{˜}{\sigma }u\\ \stackrel{˜}{\sigma }V\end{array}\right).$
(3.15)

### Theorem 3.2

Let $$c=\frac{ \Vert \rho \Vert _{\infty}}{\min_{\overline{\Omega}}\rho}$$. For any fixed $$(\mu,k,\rho(x),b(x),d(x))$$ with $$\mu>0$$, $$\Vert b \Vert _{\infty} \Vert d \Vert _{\infty}<\frac{1}{c}$$, when $$b(x)$$ is a constant function, the bifurcating solution $$(\widetilde{u}(s),\widetilde{V}(s))$$ defined by (1.10) of system (2.1) is locally asymptotically stable.

### Proof

First, we will show that 0 is the first eigenvalue of $$\widetilde{L}(0,\mu,\lambda^{*})$$. For any fixed $$(u,V)\in X$$, we obtain

$\begin{array}{rl}& L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}={\left(\begin{array}{cc}1& 0\\ -k\rho \left(x\right)\mu & 1\end{array}\right)}^{-1}{G}_{\left(u,V\right)}\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}u\\ V\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=\left(\begin{array}{cc}1& 0\\ k\rho \left(x\right)\mu & 1\end{array}\right)\left(\begin{array}{c}\mathrm{\Delta }u+\left({\lambda }^{\ast }+\mu b\right)u\\ \mathrm{\Delta }V+\mu \left(d\left(x\right)+\mu k\rho \left(x\right)\right)u-\mu V\end{array}\right)\\ & \phantom{\rule{1em}{0ex}}=\left(\begin{array}{c}\mathrm{\Delta }u+\left({\lambda }^{\ast }+\mu b\right)u\\ k\rho \left(x\right)\mu \left[\mathrm{\Delta }u+\left({\lambda }^{\ast }+\mu b\right)u\right]+\mathrm{\Delta }V+\mu \left(d\left(x\right)+\mu k\rho \left(x\right)\right)u-\mu V\end{array}\right)=0.\end{array}$
(3.16)

Taking (3.12) and (3.13) into account, we have

$L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$
(3.17)

Then 0 is the eigenvalue of $$L(0,\mu,\lambda^{*})$$. Next, we will show that 0 is the first eigenvalue of $$L(0,\mu,\lambda^{*})$$. Otherwise, then there exists a positive eigenvalue $$\widetilde{\sigma}_{1}$$ of $$L(0,\mu,\lambda^{*})$$ with the corresponding eigenfunction $$\bigl({\scriptsize\begin{matrix}{}\widetilde{u}_{1} \cr\widetilde{V}_{1} \end{matrix}} \bigr)\in X$$ such that

$L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\stackrel{˜}{u}}_{1}\\ {\stackrel{˜}{V}}_{1}\end{array}\right)=\left(\begin{array}{c}{\stackrel{˜}{\sigma }}_{1}{\stackrel{˜}{u}}_{1}\\ {\stackrel{˜}{\sigma }}_{1}{\stackrel{˜}{V}}_{1}\end{array}\right),$
(3.18)

that is,

\begin{aligned} \textstyle\begin{cases} \Delta\widetilde{u}_{1}+(\lambda^{*}+\mu b)\widetilde {u}_{1}=\widetilde{\sigma}_{1}\widetilde{u}_{1},& x\in\Omega, \\ k\rho(x)\mu[\Delta\widetilde{u}_{1}+(\lambda^{*}+\mu b)\widetilde {u}_{1}]+\Delta\widetilde{V}_{1}+\mu(d(x)+\mu k\rho(x))\widetilde {u}_{1}\\ \quad {}-\mu\widetilde{V}_{1}=\widetilde{\sigma}_{1}\widetilde {V}_{1},& x\in\Omega, \\ \partial_{\nu}\widetilde{u}_{1}=\partial_{\nu}\widetilde {V}_{1}=0,& x\in\partial\Omega. \end{cases}\displaystyle \end{aligned}
(3.19)

If $$\widetilde{u}_{1}=0$$ and $$\widetilde{V}_{1}\neq0$$ hold, the second equation of (3.19) implies

\begin{aligned}& \Delta\widetilde{V}_{1}-\mu\widetilde{V}_{1}= \widetilde{\sigma }_{1}\widetilde{V}_{1}\quad (\widetilde{ \sigma}_{1}>0), \end{aligned}
(3.20)
\begin{aligned}& \widetilde{V}_{1}=(-\Delta+\mu)^{-1}(-\widetilde{\sigma }_{1}\widetilde{V}_{1}), \end{aligned}
(3.21)

which is impossible, because $$(-\Delta+\mu)^{-1}(-\widetilde{\sigma }_{1}\widetilde{V}_{1})$$ is negative if $$-\widetilde{\sigma }_{1}\widetilde{V}_{1}$$ is negative, thus $$\widetilde{u}_{1}\neq0$$.

According to (1.5) and the scalar elliptic equation theorem, 0 is the first eigenvalue of the first equation of (3.19), which contradicts $$\widetilde{\sigma}_{1}>0$$. Then we have proved that 0 is the first eigenvalue of $$\widetilde{L}(0,\mu,\lambda^{*})$$ and the other eigenvalues are negative.

For small $$0< s<\delta$$, by Proposition I.7.2 in , there exist a perturbed eigenvalue $$\widetilde{\sigma}(s)$$ and continuous differential functions $$\varphi_{1}(s), \varphi_{2}(s) \in X\cap \operatorname{Range}(G_{(u,V)}(0,\mu,\lambda^{*}))$$ satisfying

$L\left(\stackrel{˜}{u}\left(s\right),\stackrel{˜}{V}\left(s\right),\lambda \left(s\right)\right)\left(\begin{array}{c}{\varphi }^{\ast }+{\phi }_{1}\left(s\right)\\ {\psi }^{\ast }+{\phi }_{2}\left(s\right)\end{array}\right)=\stackrel{˜}{\sigma }\left(s\right)\left(\begin{array}{c}{\varphi }^{\ast }+{\phi }_{1}\left(s\right)\\ {\psi }^{\ast }+{\phi }_{2}\left(s\right)\end{array}\right),$
(3.22)

with $$\widetilde{\sigma}(0)=0$$, $$\varphi_{1}(0)=0$$, $$\varphi_{2}(0)=0$$.

Similarly, there exist a perturbed eigenvalue $$\widetilde{\sigma }(\lambda)$$ and continuous differential functions $$\varphi_{1}(\lambda), \varphi_{2}(\lambda) \in X\cap \operatorname{Range}(G_{(u,V)}(0,\mu,\lambda^{*}))$$ satisfying

$L\left(0,\mu ,\lambda \right)\left(\begin{array}{c}{\varphi }^{\ast }+{\phi }_{1}\left(\lambda \right)\\ {\psi }^{\ast }+{\phi }_{2}\left(\lambda \right)\end{array}\right)=\stackrel{˜}{\sigma }\left(\lambda \right)\left(\begin{array}{c}{\varphi }^{\ast }+{\phi }_{1}\left(\lambda \right)\\ {\psi }^{\ast }+{\phi }_{2}\left(\lambda \right)\end{array}\right),$
(3.23)

with $$\widetilde{\sigma}(\lambda^{*})=\varphi_{1}(\lambda ^{*})=\varphi_{2}(\lambda^{*})=0$$.

Differentiation of (3.23) with respect to λ at $$\lambda^{*}$$ implies

$\frac{d}{d\lambda }L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)+L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\phi }_{1}^{\prime }\left({\lambda }^{\ast }\right)\\ {\phi }_{2}^{\prime }\left({\lambda }^{\ast }\right)\end{array}\right)={\stackrel{˜}{\sigma }}^{\prime }\left({\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),$
(3.24)

where $$\widetilde{\sigma}^{\prime}(\lambda)=\frac{d}{d\lambda }\widetilde{\sigma}(\lambda)$$.

Combining (3.1) and (3.24), we have

$〈\frac{d}{d\lambda }L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right),{l}_{2}〉={\stackrel{˜}{\sigma }}^{\prime }\left({\lambda }^{\ast }\right).$
(3.25)

It follows from (3.16) that

$\frac{d}{d\lambda }L\left(0,\mu ,{\lambda }^{\ast }\right)\left(\begin{array}{c}{\varphi }^{\ast }\\ {\psi }^{\ast }\end{array}\right)=\left(\begin{array}{c}{\varphi }^{\ast }\\ k\rho \left(x\right)\mu {\varphi }^{\ast }\end{array}\right).$
(3.26)

Using (3.1), (3.25) and (3.26), we obtain

$$\widetilde{\sigma}'\bigl(\lambda^{*}\bigr)= \int_{\Omega}\bigl(\phi^{*}\bigr)^{2}\,dx>0.$$
(3.27)

Here, $$\widetilde{\sigma}'(\lambda^{*})>0$$ implies that the semitrivial solution $$(0,\mu,\lambda^{*})$$ is stable for $$\lambda<\lambda^{*}$$ and unstable for $$\lambda>\lambda^{*}$$.

It follows from formula I.7.40 in  that

$$-\dot{\widetilde{\sigma}}(0)=\dot{\lambda}(0)\widetilde{\sigma }'\bigl(\lambda^{*}\bigr),$$
(3.28)

where $$\dot{\widetilde{\sigma}}(s)=\frac{d}{ds}\widetilde{\sigma}(s)$$.

Together with Lemma 3.1 and (3.27), we obtain

$$\dot{\widetilde{\sigma}}(0)< 0,$$
(3.29)

which yields $$\widetilde{\sigma}(s)<0$$ for small $$s>0$$, the bifurcating solution $$(\widetilde{u}(s), \widetilde{V}(s))$$ defined by (1.10) of system (2.1) is locally asymptotically stable. □

## 4 The numerical simulation results

For system (1.1), let $$\mu=-0.02$$, $$\lambda=0.022$$, $$k=100$$, $$\rho =2$$, $$b(x)=\frac{1}{2+x}$$, $$d=1$$, $$t=1000$$, $$x\in(0,1)$$ and $$(u_{0},v_{0})=(0.02+0.001\cos^{2}4\pi x, 0.001+0.001\sin4\pi x)$$ hold. We have the following numerical simulation results in order to verify the local stability of the bifurcation solutions near $$(\lambda _{*},0,\lambda_{*})$$, see Figure 1.

For system (1.1), let $$\mu=0.02$$, $$\lambda=-0.018$$, $$k=100$$, $$\rho (x)=1+x$$, $$b(x)=1$$, $$d(x)=\frac{1}{3+x}$$, $$t=1000$$, $$x\in(0,1)$$ and $$(u_{0},v_{0})=(0.001+0.001\cos^{2}4\pi x, 0.02+0.001\sin4\pi x)$$ hold. We have the following numerical simulation results which verify the local stability of the bifurcating steady states near $$(0,\mu ,\lambda^{*})$$, see Figure 2.

## 5 Conclusions

In this paper, we have studied the local stability of bifurcating solutions obtained in  of a spatially heterogeneous cooperative system with cross-diffusion. First, we give the bifurcating direction near bifurcation point. Then, using spectral analysis and the principle of exchange stability, we prove that the bifurcation solutions near the bifurcation points are locally asymptotically stable, which means that the densities of two cooperative species will tend to stabilization. Finally, we give numerical simulation results in order to verify the local stability of bifurcation solutions.

## References

1. Shigesada, N, Kawasaki, K, Teramoto, E: Spatial segregation of interacting species. J. Theor. Biol. 79, 83-99 (1979)

2. Ni, WM, Wu, YP, Xu, Q: The existence and stability of nontrivial steady states for S-K-T competition model with cross-diffusion. Discrete Contin. Dyn. Syst. 34, 5271-5298 (2014)

3. Wang, L, Wu, YP, Xu, Q: Instability of spiky steady states for S-K-T biological competing model with cross-diffusion. Nonlinear Anal. 159, 424-457 (2017)

4. Wu, YP: The instability of spiky steady states for a competing species model with cross-diffusion. J. Differ. Equ. 213, 289-340 (2005)

5. Kadota, T, Kuto, K: Positive steady-states for a prey-predator model with some nonlinear diffusion terms. J. Math. Anal. Appl. 323, 1387-1401 (2006)

6. Xu, Q, Guo, Y: The existence and stability of steady states for a prey-predator system with cross diffusion of quasilinear fractional type. Acta Math. Appl. Sin. 30, 257-270 (2014)

7. Du, Y, Peng, R, Wang, MX: Effect of a protection zone in the diffusive Leslie predator-prey model. J. Differ. Equ. 246, 3932-3956 (2009)

8. Du, Y, Shi, JP: A diffusive predator-prey model with a protection zone. J. Differ. Equ. 229, 63-91 (2006)

9. Du, Y, Shi, JP: Some recent results on diffusive predator-prey models in spatially heterogeneous environment. Fields Inst. Commun. 48, 1-41 (2006)

10. Hutson, V, Lou, Y, Mischaikow, K: Spatial heterogeneity of resources versus Lotka-Volterra dynamics. J. Differ. Equ. 185, 97-136 (2002)

11. Hutson, V, Lou, Y, Mischaikow, K: Convergence in competition models with small diffusion coefficients. J. Differ. Equ. 211, 135-161 (2005)

12. Hutson, V, Lou, Y, Mischaikow, K, Polác̆ik, P: Competing species near a degenerate limit. SIAM J. Math. Anal. 34, 453-491 (2002)

13. Hutson, V, Mischaikow, K, Polác̆ik, P: The evolution of dispersal rates in a heterogeneous time-periodic environment. J. Math. Biol. 43, 501-533 (2001)

14. Wang, YX, Li, WT: Fish-hook shaped global bifurcation branch of a spatially heterogeneous cooperative system with cross-diffusion. J. Differ. Equ. 251, 1670-1695 (2001)

15. Li, WT, Wang, YX, Zhang, JF: Stability of positive stationary solutions to a spatially heterogeneous cooperative system with cross-diffusion. Electron. J. Differ. Equ. 2012, 223 (2012)

16. Shi, JP: Persistence and bifurcation of degenerate solutions. J. Funct. Anal. 169, 494-531 (1999)

17. Kielhöfer, H: Bifurcation Theory: An Introduction with Applications to PDEs. Springer, Berlin (2004)

## Acknowledgements

This work was supported by the National Natural Science Foundation of China (No.11501031, No.11471221).

## Author information

Authors

### Contributions

All authors contributed equally to the writing of this paper. All authors read and approved the final manuscript.

### Corresponding author

Correspondence to Guangping Chang.

## Ethics declarations

### Competing interests

The authors declare that they have no competing interests. 