On a coupled system of higher order nonlinear Caputo fractional differential equations with coupled Riemann–Stieltjes type integro-multipoint boundary conditions

Abstract

We study a coupled system of Caputo fractional differential equations with coupled non-conjugate Riemann–Stieltjes type integro-multipoint boundary conditions. Existence and uniqueness results for the given boundary value problem are obtained by applying the Leray–Schauder nonlinear alternative, the Krasnoselskii fixed point theorem and Banach’s contraction mapping principle. Examples are constructed to illustrate the obtained results.

1 Introduction

Fractional-order differential systems constitute the mathematical models of many real world problems. Examples include disease models [1,2,3], anomalous diffusion [4, 5], synchronization of chaotic systems [6, 7], ecological models [8]. For applications in bioengineering, chaos and financial economics, we refer the reader to [9,10,11]. The details about rheological models in the context of local fractional derivatives can be found in [12]. In view of the extensive applications of such systems, many researchers turned to investigation of the theoretical aspects of fractional differential equations. In particular, there was a special attention on proving the existence and uniqueness of solutions for fractional differential systems supplemented with a variety of classical and non-classical (nonlocal) boundary conditions with the aid of modern methods of functional analysis. For details and examples, see [13,14,15,16,17,18,19,20,21,22] and the references cited therein. It is imperative to mention that fractional-order models are more practical and informative than their integer-order counterparts. It has been mainly due to the fact that fractional-order operators can describe the hereditary properties of the processes and phenomena under investigation.

In this paper, we study the existence of solutions for a nonlinear coupled system of Liouville–Caputo type fractional differential equations on an arbitrary domain:

$$ \textstyle\begin{cases} ^{c}D^{\alpha }x(t)=f(t,x(t),y(t)),& 3< \alpha \leq 4, t\in [a,b], \\ ^{c}D^{\beta }y(t)=g(t,x(t),y(t)),& 3< \beta \leq 4, t\in [a,b], \end{cases} $$

(1.1)

equipped with coupled non-conjugate Riemann–Stieltjes integro-multipoint boundary conditions:

$$ \textstyle\begin{cases} x'(a)=0,\qquad x(b)=0,\qquad x'(b)=0, \\ x(a) = \int _{a}^{b}y(s)\,dA(s)+\sum_{i=1}^{n-2} {\alpha _{i}y(\xi _{i})} , \\ y'(a)=0,\qquad y(b)=0,\qquad y'(b)=0, \\ y(a) = \int _{a}^{b}x(s)\,dA(s)+\sum_{i=1}^{n-2} {\beta _{i}x(\xi _{i})}, \end{cases} $$

(1.2)

where $^{c}D^{\varrho }$ denotes the Caputo fractional derivative of order ϱ with ($\varrho =\alpha ,\beta$), $f,g:[a,b]\times \mathbb{R}\times \mathbb{R}\rightarrow \mathbb{R}$ are given continuous functions, $a<\xi _{1}<\xi _{2}< \cdots <\xi _{n-2}<b$, $\alpha _{i}, \beta _{i}\in \mathbb{R}, i=1,2,\ldots , n-2$ and A is a function of bounded variation. In passing we remark that the present work is motivated by a recent paper [23], where the authors studied the existence and stability of solutions for a fractional-order differential equation with non-conjugate Riemann–Stieltjes integro-multipoint boundary conditions.

We arrange the rest of the paper as follows. Section 2 contains an auxiliary result that plays a key role in analyzing the given problem. Existence results for the problem (1.1)–(1.2) with the illustrative examples are presented in Sect. 3, while the uniqueness of solutions is discussed in Sect. 4.

2 Auxiliary result

Before giving an auxiliary result for system (1.1)–(1.2), we recall some necessary definitions of fractional calculus [24, 25].

Definition 2.1

Let ϑ be a locally integrable real-valued function on $-\infty \leq a< s< b\leq +\infty$. The Riemann–Liouville fractional integral $I^{\vartheta }_{a}$ of order $\vartheta \in \mathbb{R}\ ( \vartheta >0)$ for the function q is defined as

$$ I^{\vartheta }_{a}q(s)=(q*K_{\vartheta }) (s)= \frac{1}{\varGamma (\vartheta )} \int _{a}^{s} (s-u)^{\vartheta -1}q(u)\,du, $$

where $K_{\vartheta }=\frac{t^{\vartheta -1}}{\varGamma (\vartheta )}, \varGamma$ denotes the Euler gamma function.

Definition 2.2

The Caputo derivative of fractional order ϑ for an $(m-1)$-times absolutely continuous function $q:{[a,\infty )}\longrightarrow \mathbb{R}$ is defined as

$$ ^{c}D^{\vartheta }q(s)=\frac{1}{\varGamma (m-\vartheta )} \int _{a}^{s}(s-u)^{m- \vartheta -1}q^{(m)}(u) \,du,\quad m-1< \vartheta \leq m, m=[\vartheta ]+1, $$

where $[\vartheta ]$ denotes the integer part of the real number ϑ.

Lemma 2.3

([24])

The general solution of the fractional differential equation $^{c}D^{\vartheta }x(s)=0, m-1<\vartheta <m, s\in [a,b]$, is

$$ x(s)=\upsilon _{0}+\upsilon _{1}(s-a)+\upsilon _{2}(s-a)^{2}+\cdots + \upsilon _{m-1}(s-a)^{m-1}, $$

where $\upsilon _{i}\in \mathbb{R}, i=0,1,\ldots ,m-1$. Furthermore,

$$ I^{\vartheta } {}^{c}D^{\vartheta }x(s)=x(s)+\sum _{i=0}^{m-1}{\upsilon _{i}(s-a)^{i}}. $$

Lemma 2.4

For $\widehat{f},\widehat{g} \in C([a,b],{\mathbb{R}})$, the solution of the linear system of fractional differential equations:

$$ \textstyle\begin{cases} ^{c}D^{\alpha }x(t)=\widehat{f}(t),& 3< \alpha \leq 4, t\in [a,b], \\ ^{c}D^{\beta }y(t)=\widehat{g}(t),& 3< \beta \leq 4, t\in [a,b], \end{cases} $$

(2.1)

supplemented with the boundary conditions (1.2) is equivalent to the system of integral equations:

$$\begin{aligned} x(t) = {}& \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{\varGamma (\alpha )} \widehat{f}(s)\,ds-\phi _{1}(t) \int _{a}^{b}\frac{(b-s)^{\alpha -1}}{ \varGamma (\alpha )}\widehat{f}(s) \,ds -\phi _{6}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -2}}{\varGamma (\alpha -1)}\widehat{f}(s) \,ds \\ &{}+ \phi _{2}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )}\widehat{f}(u) \,du \biggr) \,dA(s)+\sum_{i=1}^{n-2} { \beta _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma ( \alpha )} \widehat{f}(s)\,ds \Biggr] \\ &{}-\phi _{3}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds-\phi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{ \varGamma (\beta -1)}\widehat{g}(s) \,ds \\ &{}+\phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}\widehat{g}(u) \,du \biggr)\,dA(s) \\ &{}+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds \Biggr], \end{aligned}$$

(2.2)

$$\begin{aligned} y(t) = {}& \int _{a}^{t}\frac{(t-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds-\psi _{1}(t) \int _{a}^{b}\frac{(b-s)^{\alpha -1}}{ \varGamma (\alpha )}\widehat{f}(s) \,ds -\psi _{2}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -2}}{\varGamma (\alpha -1)}\widehat{f}(s) \,ds \\ &{}+ \phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )}\widehat{f}(u) \,du \biggr) \,dA(s)+\sum_{i=1}^{n-2} { \beta _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma ( \alpha )} \widehat{f}(s)\,ds \Biggr] \\ &{}-\psi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds-\psi _{5}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{ \varGamma (\beta -1)}\widehat{g}(s) \,ds \\ &{}+\psi _{3}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}\widehat{g}(u) \,du \biggr) \,dA(s) \\ &{}+\sum_{i=1}^{n-2}{ \alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds \Biggr], \end{aligned}$$

(2.3)

where

$$\begin{aligned} &\phi _{i}(t) = \nu _{i}+(t-a)^{2}\mu _{i}+(t-a)^{3}\delta _{i},\quad i=1,\ldots ,6, \end{aligned}$$

(2.4)

$$\begin{aligned} &\psi _{j}(t) = \omega _{j}+(t-a)^{2} \rho _{j}+(t-a)^{3}\lambda _{j},\quad j=1,\ldots ,5, \end{aligned}$$

(2.5)

$$\begin{aligned} &\mu _{k} = \frac{-3(b-a)\delta _{k}}{2},\quad k=1,2,3,4,5,\qquad \mu _{6}= \frac{1-3(b-a)^{2}\delta _{6}}{2(b-a)}, \end{aligned}$$

(2.6)

$$\begin{aligned} &\rho _{m} = \frac{-3(b-a)\lambda _{m}}{2},\quad m=1,2,3,4,\qquad \rho _{5}= \frac{1-3(b-a)^{2}\lambda _{5}}{2(b-a)}, \end{aligned}$$

(2.7)

$$\begin{aligned} &\delta _{1} = \frac{2\nu _{1}-2}{(b-a)^{3}}, \delta _{n}= \frac{2\nu _{n}}{(b-a)^{3}},\quad n=2,3,4,5,\qquad \delta _{6}= \frac{2\nu _{6}+(b-a)}{(b-a)^{3}}, \end{aligned}$$

(2.8)

$$\begin{aligned} &\lambda _{r} = \frac{2\omega _{r}}{(b-a)^{3}},\quad r=1,2,3,\qquad \lambda _{4}= \frac{2\omega _{4}-2}{(b-a)^{3}}, \qquad\lambda _{5}= \frac{2\omega _{5}+(b-a)}{(b-a)^{3}}, \end{aligned}$$

(2.9)

$$\begin{aligned} &\textstyle\begin{cases} \nu _{1}= \nu _{2} (A_{4}-\frac{2\gamma _{3}}{(b-a)^{3}} ),\qquad \nu _{2}= \frac{2(b-a)^{3}\gamma _{1}}{(b-a)^{6}-4\gamma _{1}\gamma _{3}}, \\ \nu _{3}= \nu _{5} (A_{1}-\frac{2\gamma _{1}}{(b-a)^{3}} ),\qquad \nu _{4}= \nu _{5} (\gamma _{2}+\frac{\gamma _{1}}{(b-a)^{2}} ), \\ \nu _{5}= \frac{(b-a)^{6}}{(b-a)^{6}-4\gamma _{1}\gamma _{3}},\qquad \nu _{6}= \nu _{2} (\gamma _{4}+\frac{\gamma _{3}}{(b-a)^{2}} ), \end{cases}\displaystyle \end{aligned}$$

(2.10)

$$\begin{aligned} & \textstyle\begin{cases} \omega _{1}= \nu _{5} (A_{4}-\frac{2\gamma _{3}}{(b-a)^{3}} ),\qquad \omega _{2}= \nu _{5} (\gamma _{4}+\frac{\gamma _{3}}{(b-a)^{2}} ), \\ \omega _{3}= \frac{2(b-a)^{3}\gamma _{3}}{(b-a)^{6}-4\gamma _{1}\gamma _{3}},\qquad \omega _{4}= \omega _{3} (A_{1}-\frac{2\gamma _{1}}{(b-a)^{3}} ), \\ \omega _{5}= \omega _{3} (\gamma _{2}+\frac{\gamma _{1}}{(b-a)^{2}} ), \end{cases}\displaystyle \end{aligned}$$

(2.11)

$$\begin{aligned} & \textstyle\begin{cases} \gamma _{1}= \frac{(b-a)^{3}A_{1}-3(b-a)A_{2}+2A_{3}}{2}, \qquad\gamma _{2}= \frac{A_{2}-(b-a)^{2}A_{1}}{2(b-a)}, \\ \gamma _{3}= \frac{(b-a)^{3}A_{4}-3(b-a)A_{5}+2A_{6}}{2},\qquad \gamma _{4}= \frac{A_{5}-(b-a)^{2}A_{4}}{2(b-a)}, \end{cases}\displaystyle \end{aligned}$$

(2.12)

$$\begin{aligned} & \textstyle\begin{cases} A_{1}= \int _{a}^{b}\,dA(s)+\sum_{i=1}^{n-2}{\alpha _{i}},\qquad A_{2}= \int _{a}^{b}(s-a)^{2}\,dA(s)+\sum_{i=1}^{n-2}{\alpha _{i}}(\xi _{i}-a)^{2}, \\ A_{3}= \int _{a}^{b}(s-a)^{3}\,dA(s)+\sum_{i=1}^{n-2}{\alpha _{i}}(\xi _{i}-a)^{3}, \qquad A_{4}= \int _{a}^{b}\,dA(s)+\sum_{i=1}^{n-2}{\beta _{i}}, \\ A_{5}= \int _{a}^{b}(s-a)^{2}\,dA(s)+\sum_{i=1}^{n-2}{\beta _{i}}(\xi _{i}-a)^{2}, \\ A_{6}= \int _{a}^{b}(s-a)^{3}\,dA(s)+\sum_{i=1}^{n-2}{\beta _{i}}(\xi _{i}-a)^{3}. \end{cases}\displaystyle \end{aligned}$$

(2.13)

Proof

By Lemma 2.3, the general solutions of fractional differential equations in (2.1) can be written as

$$\begin{aligned} &x(t)= \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{\varGamma (\alpha )} \widehat{f}(s)\,ds+c _{0}+c_{1}(t-a) +c_{2}(t-a)^{2}+c_{3}(t-a)^{3}, \end{aligned}$$

(2.14)

$$\begin{aligned} &y(t)= \int _{a}^{t}\frac{(t-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds+b _{0}+b_{1}(t-a) +b_{2}(t-a)^{2}+b_{3}(t-a)^{3}, \end{aligned}$$

(2.15)

where $c_{i},b_{i} \in \mathbb{R}, i=0,1,2,3$ are unknown arbitrary constants.

Using the boundary conditions (1.2) in (2.14), (2.15), we obtain $c_{1}=0, b_{1}=0$, and

$$\begin{aligned} & c_{0}+(b-a)^{2}c_{2}+(b-a)^{3}c_{3}=I_{1}, \end{aligned}$$

(2.16)

$$\begin{aligned} &b_{0}+(b-a)^{2}b_{2}+(b-a)^{3}b_{3}=K_{1}, \end{aligned}$$

(2.17)

$$\begin{aligned} & 2(b-a)c_{2}+3(b-a)^{2}c_{3}=I_{2}, \end{aligned}$$

(2.18)

$$\begin{aligned} & 2(b-a)b_{2}+3(b-a)^{2}b_{3}=K_{2}, \end{aligned}$$

(2.19)

$$\begin{aligned} & c_{0}-A_{1}b_{0}-A_{2}b_{2}-A_{3}b_{3}=K_{3}, \end{aligned}$$

(2.20)

$$\begin{aligned} & b_{0}-A_{4}c_{0}-A_{5}c_{2}-A_{6}c_{3}=I_{3}, \end{aligned}$$

(2.21)

where $A_{i}\ (i=1,\ldots ,6)$ are given by (2.13) and $I_{i},K_{i}\ (i=1,2,3)$ are defined by

$$\begin{aligned} &I_{1} =- \int _{a}^{b}\frac{(b-s)^{\alpha -1}}{\varGamma (\alpha )} \widehat{f}(s)\,ds,\qquad I_{2}=- \int _{a}^{b}\frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)} \widehat{f}(s)\,ds, \\ &I_{3} = \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )}\widehat{f}(u) \,du \biggr) \,dA(s)+\sum_{i=1}^{n-2}{\beta _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )} \widehat{f}(s)\,ds, \\ &K_{1} =- \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds, \qquad K_{2}=- \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma ( \beta -1)}\widehat{g}(s) \,ds, \\ &K_{3} = \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma ( \beta )}\widehat{g}(u) \,du \biggr)\,dA(s)+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\beta -1}}{\varGamma (\beta )} \widehat{g}(s)\,ds. \end{aligned}$$

From (2.16) and (2.18), we find that

$$ c_{3}=\frac{2}{(b-a)^{3}} \biggl(c_{0}-I_{1}+ \frac{(b-a)}{2}I_{2} \biggr). $$

(2.22)

Eliminating $b_{2}$ from (2.17) and (2.19), we get

$$ b_{3}=\frac{2}{(b-a)^{3}} \biggl(b_{0}-K_{1}+ \frac{(b-a)}{2}K_{2} \biggr). $$

(2.23)

Combining (2.19), (2.20) and (2.23) yields

$$ c_{0}=\gamma _{1}b_{3}+A_{1}K_{1}+ \gamma _{2}K_{2}+K_{3}. $$

(2.24)

From (2.18), (2.21) and (2.22), we have

$$ b_{0}=\gamma _{3}c_{3}+A_{4}I_{1}+ \gamma _{4}I_{2}+I_{3}, $$

(2.25)

where $\gamma _{i}\ (i=1,\ldots ,4)$ are given by (2.12).

Eliminating $b_{3}$ from (2.23) and (2.24), and $c_{3}$ from (2.22) and (2.25), we obtain

$$\begin{aligned} & c_{0}=\frac{2\gamma _{1}}{(b-a)^{3}}b_{0}+ \biggl(A_{1}- \frac{2\gamma _{1}}{(b-a)^{3}} \biggr)K_{1}+ \biggl(\gamma _{2}+\frac{\gamma _{1}}{(b-a)^{2}} \biggr)K_{2}+K_{3}, \end{aligned}$$

(2.26)

$$\begin{aligned} & b_{0}=\frac{2\gamma _{3}}{(b-a)^{3}}c_{0}+ \biggl(A_{4}- \frac{2\gamma _{3}}{(b-a)^{3}} \biggr)I_{1}+ \biggl(\gamma _{4}+\frac{\gamma _{3}}{(b-a)^{2}} \biggr)I_{2}+I_{3}. \end{aligned}$$

(2.27)

Solving (2.26) and (2.27) simultaneously for $c_{0}$ and $b_{0}$, we obtain

$$\begin{aligned} & c_{0}=\nu _{1}I_{1}+\nu _{6}I_{2}+\nu _{2}I_{3}+\nu _{3}K_{1}+\nu _{4}K _{2}+\nu _{5}K_{3}, \end{aligned}$$

(2.28)

$$\begin{aligned} &b_{0}=\omega _{1}I_{1}+ \omega _{2}I_{2}+\nu _{5}I_{3}+ \omega _{4}K_{1}+ \omega _{5}K_{2}+ \omega _{3}K_{3}, \end{aligned}$$

(2.29)

where $\nu _{i}\ (i=1,\ldots ,6)$ and $\omega _{i}\ (i=1,\ldots ,5)$ are defined by (2.10) and (2.11), respectively. Using (2.28) in (2.22) and (2.29) in (2.23), we get

$$\begin{aligned} &c_{3}=\delta _{1}I_{1}+\delta _{6}I_{2}+\delta _{2}I_{3}+ \delta _{3}K_{1}+ \delta _{4}K_{2}+ \delta _{5}K_{3}, \\ &b_{3}=\lambda _{1}I_{1}+\lambda _{2}I_{2}+\delta _{5}I_{3}+ \lambda _{4}K _{1}+\lambda _{5}K_{2}+ \lambda _{3}K_{3}, \end{aligned}$$

where $\delta _{i}\ (i=1,\ldots ,6)$ and $\lambda _{i}\ (i=1,\ldots ,5)$ are given by (2.8) and (2.9), respectively.

Substituting the values of $c_{3}$ and $b_{3}$ in (2.18) and 2.19), respectively, we find that

$$\begin{aligned} &c_{2}=\mu _{1}I_{1}+\mu _{6}I_{2}+\mu _{2}I_{3}+\mu _{3}K_{1}+\mu _{4}K _{2}+\mu _{5}K_{3}, \\ &b_{2}=\rho _{1}I_{1}+\rho _{2}I_{2}+\mu _{5}I_{3}+\rho _{4}K_{1}+\rho _{5}K _{2}+ \rho _{3}K_{3}, \end{aligned}$$

where $\mu _{i}\ (i=1,\ldots ,6)$ and $\rho _{i}\ (i=1,\ldots ,5)$ are, respectively, defined by (2.6) and (2.7). Inserting the values of $c_{0},c_{1},c_{2},c_{3}$ in (2.14) and $b_{0},b_{1},b_{2}, b_{3}$ in (2.15), we obtain (2.2) and (2.3). By direct computation, one can obtain the converse. The proof is complete. □

3 Existence results

We define space $\mathcal{A}=\{x|x\in C([a,b],\mathbb{R})\}$ equipped with the norm $\|x\|= \sup_{t\in [a,b]}|x(t)|$. Obviously $(\mathcal{A},\|\cdot \|)$ is a Banach space and consequently, the product space $(\mathcal{A}\times \mathcal{A},\|\cdot \|)$ is a Banach space with norm $\|(x,y)\|=\|x\|+ \|y\|$ for $(x,y)\in \mathcal{A}\times \mathcal{A}$.

In view of Lemma 2.4, we define an operator $\mathcal{F}: \mathcal{A}\times \mathcal{A}\rightarrow \mathcal{A}\times \mathcal{A}$ as

$$ \mathcal{F}(x,y) (t):=\bigl(\mathcal{F}_{1}(x,y) (t),\mathcal{F}_{2}(x,y) (t)\bigr), $$

(3.1)

where

$$\begin{aligned} \mathcal{F}_{1}(x,y) (t) = {}& \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{ \varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds-\phi _{1}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}-\phi _{6}(t) \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}f \bigl(s,x(s),y(s)\bigr)\,ds+ \phi _{2}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{ \varGamma (\alpha )} \\ &{}\times f\bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s) +\sum _{i=1}^{n-2}{\beta _{i}} \int _{a} ^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \Biggr] \\ &{}-\phi _{3}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}- \phi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}+\phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}g \bigl(u,x(u),y(u)\bigr)\,du \biggr)\,dA(s) \\ &{}+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \Biggr], \end{aligned}$$

(3.2)

$$\begin{aligned} \mathcal{F}_{2}(x,y) (t) = {}& \int _{a}^{t}\frac{(t-s)^{\beta -1}}{ \varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds-\psi _{1}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}-\psi _{2}(t) \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}f \bigl(s,x(s),y(s)\bigr)\,ds +\phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{ \varGamma (\alpha )} \\ &{}\times f\bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s)+\sum _{i=1}^{n-2}{\beta _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \Biggr] \\ &{}-\psi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds\\ &{}- \psi _{5}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}+\psi _{3}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}g \bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s) \\ &{}+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \Biggr], \end{aligned}$$

(3.3)

and $\phi _{i}(t), i=1,\ldots ,6$ and $\psi _{j}(t), j=1,\ldots ,5$ are given by (2.4) and (2.5), respectively.

In the forthcoming analysis, we assume that $f,g:[a,b]\times \mathbb{R}\times \mathbb{R}\rightarrow \mathbb{R}$ are continuous functions satisfying the following conditions:

($\mathcal{O}_{1}$):

$\forall t\in [a,b]$ and $x_{j},y_{j} \in \mathbb{R}, j=1,2$, there exist $L_{i}, i=1,2$ such that

$$\begin{aligned} &\bigl\vert f(t,x_{1},y_{1})-f(t,x_{2},y_{2}) \bigr\vert \leq L_{1} \bigl( \vert x_{1}-x_{2} \vert + \vert y_{1}-y _{2} \vert \bigr), \\ &\bigl\vert g(t,x_{1},y_{1})-g(t,x_{2},y_{2}) \bigr\vert \leq L_{2} \bigl( \vert x_{1}-x_{2} \vert + \vert y_{1}-y _{2} \vert \bigr); \end{aligned}$$

($\mathcal{O}_{2}$):

$\forall t\in [a,b], x,y\in \mathbb{R}$ there exist real constants $\varepsilon _{i}, \kappa _{i}\geq 0, i=1,2, \varepsilon _{0},\kappa _{0}>0$ such that

$$\begin{aligned} &\bigl\vert f(t,x,y) \bigr\vert \leq \varepsilon _{0}+ \varepsilon _{1} \vert x \vert +\varepsilon _{2} \vert y \vert , \\ &\bigl\vert g(t,x,y) \bigr\vert \leq \kappa _{0}+ \kappa _{1} \vert x \vert +\kappa _{2} \vert y \vert . \end{aligned}$$

For computational convenience, we introduce the following notations:

$$\begin{aligned} \begin{aligned} &\varLambda _{1} = \frac{(b-a)^{\alpha }}{\varGamma (\alpha +1)}+ \widetilde{\phi }_{1}\frac{(b-a)^{\alpha }}{\varGamma (\alpha +1)}+ \widetilde{\phi }_{6}\frac{(b-a)^{\alpha -1}}{\varGamma (\alpha )}\\ &\phantom{\varLambda _{1} =}{}+ \widetilde{\phi }_{2} \Biggl( \int _{a}^{b}\frac{(s-a)^{\alpha }}{\varGamma ( \alpha +1)}\,dA(s)+\sum _{i=1}^{n-2}{ \vert \beta _{i} \vert }\frac{(\xi _{i}-a)^{ \alpha }}{\varGamma (\alpha +1)} \Biggr), \\ &\varLambda _{2} = \widetilde{\phi }_{3} \frac{(b-a)^{\beta }}{\varGamma ( \beta +1)}+\widetilde{\phi }_{4}\frac{(b-a)^{\beta -1}}{\varGamma ( \beta )}\\ &\phantom{\varLambda _{2} =}{}+ \widetilde{\phi }_{5} \Biggl( \int _{a}^{b}\frac{(s-a)^{\beta }}{ \varGamma (\beta +1)}\,dA(s)+\sum _{i=1}^{n-2}{ \vert \alpha _{i} \vert }\frac{(\xi _{i}-a)^{ \beta }}{\varGamma (\beta +1)} \Biggr), \\ &\varLambda _{3} =\widetilde{\psi }_{1} \frac{(b-a)^{\alpha }}{\varGamma ( \alpha +1)}+\widetilde{\psi }_{2}\frac{(b-a)^{\alpha -1}}{\varGamma ( \alpha )}\\ &\phantom{\varLambda _{3} =}{}+ \widetilde{\phi }_{5} \Biggl( \int _{a}^{b}\frac{(s-a)^{\alpha }}{ \varGamma (\alpha +1)}\,dA(s)+\sum _{i=1}^{n-2}{ \vert \beta _{i} \vert }\frac{(\xi _{i}-a)^{ \alpha }}{\varGamma (\alpha +1)} \Biggr), \\ &\varLambda _{4} =\frac{(b-a)^{\beta }}{\varGamma (\beta +1)}+ \widetilde{\psi }_{4}\frac{(b-a)^{\beta }}{\varGamma (\beta +1)}+ \widetilde{\psi }_{5} \frac{(b-a)^{\beta -1}}{\varGamma (\beta )}\\ &\phantom{\varLambda _{4} =}{}+ \widetilde{\psi }_{3} \Biggl( \int _{a}^{b}\frac{(s-a)^{\beta }}{\varGamma ( \beta +1)}\,dA(s) +\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert }\frac{(\xi _{i}-a)^{\beta }}{\varGamma ( \beta +1)} \Biggr), \end{aligned} \end{aligned}$$

(3.4)

where $\widetilde{\phi }_{i}= \sup_{t\in [a,b]}|\phi _{i}(t)|, i=1,\ldots ,6$ and $\widetilde{\psi } _{j}= \sup_{t\in [a,b]}|\psi _{j}(t)|, j=1,\ldots ,5$,

$$\begin{aligned} &N_{1} = \sup_{t\in [a,b]} \bigl\vert f(t,0,0) \bigr\vert < \infty ,\qquad N_{2}= \sup _{t\in [a,b]} \bigl\vert g(t,0,0,) \bigr\vert < \infty , \end{aligned}$$

(3.5)

$$\begin{aligned} \begin{aligned} & \Delta =\varLambda _{1}L_{1}+ \varLambda _{2}L_{2},\qquad \overline{\Delta }= \varLambda _{3}L_{1}+\varLambda _{4}L_{2},\\ & M=\varLambda _{1}N_{1}+\varLambda _{2}N_{2},\qquad \overline{M}=\varLambda _{3}N_{1}+\varLambda _{4}N_{2}, \end{aligned} \end{aligned}$$

(3.6)

$$\begin{aligned} &\varOmega _{0}=(\varLambda _{1}+ \varLambda _{3})\varepsilon _{0}+(\varLambda _{2}+ \varLambda _{4})\kappa _{0}, \end{aligned}$$

(3.7)

$$\begin{aligned} \begin{aligned} &\varOmega _{1}=(\varLambda _{1}+ \varLambda _{3})\varepsilon _{1}+(\varLambda _{2}+ \varLambda _{4})\kappa _{1},\qquad \varOmega _{2}=(\varLambda _{1}+\varLambda _{3}) \varepsilon _{2}+(\varLambda _{2}+ \varLambda _{4})\kappa _{2},\\ & \varOmega = \max \{ \varOmega _{1},\varOmega _{2}\}. \end{aligned} \end{aligned}$$

(3.8)

Now we present our main results. The first result, based on the Leray–Schauder alternative, deals with the existence of solution for system (1.1)–(1.2).

Lemma 3.1

(Leray–Schauder alternative [26])

Let $\mathfrak{J}:{\mathcal{U}}\longrightarrow \mathcal{U}$be a completely continuous operator (i.e., a map that restricted to any bounded set in $\mathcal{U}$is compact). Let $\mathcal{Q}( \mathfrak{J})=\{x\in \mathcal{U}:x=\eta \mathfrak{J}(x)\textit{ for some }0<\eta <1\}$. Then either the set $\mathcal{Q}(\mathfrak{J})$is unbounded, or $\mathfrak{J}$has at lest one fixed point.

Theorem 3.2

Assume that $f,g:[a,b]\times \mathbb{R}\times \mathbb{R}\rightarrow \mathbb{R}$are continuous functions satisfying assumption ($\mathcal{O}_{2}$). Then system (1.1)–(1.2) has at least one solution on $[a,b]$if $\varOmega <1$, whereΩis given by (3.8).

Proof

In the first step, we show that the operator $\mathcal{F}:\mathcal{A}\times \mathcal{A}\rightarrow \mathcal{A} \times \mathcal{A}$ is completely continuous. Notice that the operator $\mathcal{F}$ is continuous in view of continuity of the functions f and g.

Let $\mathcal{V} \subset \mathcal{A}\times \mathcal{A}$ be bounded. Then there exist positive constants $\theta _{1}$ and $\theta _{2}$ such that $|f(t,x(t),y(t))|\leq \theta _{1}, |g(t,x(t),y(t))|\leq \theta _{2}, \forall (x,y)\in \mathcal{V}$. So, for any $(x,y)\in \mathcal{V}$, we have

$$\begin{aligned} \bigl\vert \mathcal{F}_{1}(x,y) (t) \bigr\vert \leq{} & \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{ \varGamma (\alpha )}\theta _{1}\,ds+ \bigl\vert \phi _{1}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )}\theta _{1}\,ds \\ &{}+ \bigl\vert \phi _{6}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}\theta _{1}\,ds+ \bigl\vert \phi _{2}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma ( \alpha )}\theta _{1} \biggr) \,dA(s) \\ &{} +\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )}\theta _{1}\,ds \Biggr] \\ &{}+ \bigl\vert \phi _{3}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \theta _{2}\,ds+ \bigl\vert \phi _{4}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}\theta _{2}\,ds \\ &{}+ \bigl\vert \phi _{5}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )}\theta _{2}\,du \biggr)\,dA(s)+\sum_{i=1}^{n-2}{ \alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\beta -1}}{\varGamma (\beta )} \theta _{2}\,ds \Biggr] \\ \leq{} & \varLambda _{1} \theta _{1} + \varLambda _{2} \theta _{2}. \end{aligned}$$

Thus,

$$\begin{aligned} \bigl\Vert \mathcal{F}_{1}(x,y) \bigr\Vert \leq \varLambda _{1} \theta _{1} + \varLambda _{2} \theta _{2}. \end{aligned}$$

(3.9)

Similarly, we can get

$$\begin{aligned} \bigl\Vert \mathcal{F}_{2}(x,y) \bigr\Vert \leq \varLambda _{3} \theta _{1} + \varLambda _{4} \theta _{2}. \end{aligned}$$

(3.10)

Hence, from (3.9) and (3.10), it follows that $\mathcal{F}$ is uniformly bounded, since $\|\mathcal{F}(x,y)\|\leq ( \varLambda _{1}+\varLambda _{3})\theta _{1}+(\varLambda _{2} +\varLambda _{4})\theta _{2}$.

Next, we show that the operator $\mathcal{F}$ is equicontinuous. For $t_{1},t_{2} \in [a,b]$ with $t_{1}< t_{2}$, we obtain

$$\begin{aligned} & \bigl\vert \mathcal{F}_{1}(x,y) (t_{2})- \mathcal{F}_{1}(x,y) (t_{1}) \bigr\vert \\ &\quad \leq \biggl\vert \int _{a}^{t_{1}}\frac{[(t_{2}-s)^{\alpha -1}-(t_{1}-s)^{ \alpha -1}]}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ &\qquad{} + \biggl\vert \int _{t _{1}}^{t_{2}}\frac{(t_{2}-s)^{\alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ &\qquad{}+ \bigl\vert \phi _{1}(t_{2})-\phi _{1}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{6}(t_{2})-\phi _{6}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \alpha -2}}{\varGamma (\alpha -1)} \bigl\vert f\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{2}(t_{2})-\phi _{2}(t_{1}) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(u,x(u),y(u)\bigr) \bigr\vert \,du \biggr) \,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \Biggr] \\ &\qquad{}+ \bigl\vert \phi _{3}(t_{2})-\phi _{3}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{4}(t_{2})-\phi _{4}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \beta -2}}{\varGamma (\beta -1)} \bigl\vert g\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{5}(t_{2})-\phi _{5}(t_{1}) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(u,x(u),y(u)\bigr) \bigr\vert \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x(s),y(s)\bigr) \bigr\vert \,ds \Biggr] \\ &\quad\leq \frac{\theta _{1}}{\varGamma (\alpha +1)} \bigl[2(t_{2}-t_{1})^{ \alpha }+ \bigl\vert (t_{2}-a)^{\alpha }-(t_{1}-a)^{\alpha } \bigr\vert \bigr] \\ &\qquad{}+\theta _{1} \bigl\vert \phi _{1}(t_{2})- \phi _{1}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )} \,ds+\theta _{1} \bigl\vert \phi _{6}(t_{2})-\phi _{6}(t_{1}) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)} \,ds \\ &\qquad{}+\theta _{1} \bigl\vert \phi _{2}(t_{2})- \phi _{2}(t_{1}) \bigr\vert \Biggl[ \int _{a} ^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )} \,du \biggr) \,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )} \,ds \Biggr]+\theta _{2} \bigl\vert \phi _{3}(t_{2})- \phi _{3}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma ( \beta )} \,ds \\ &\qquad{}+\theta _{2} \bigl\vert \phi _{4}(t_{2})- \phi _{4}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \beta -2}}{\varGamma (\beta -1)} \,ds \\ &\qquad{}+\theta _{2} \bigl\vert \phi _{5}(t_{2})- \phi _{5}(t_{1}) \bigr\vert \Biggl[ \int _{a} ^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )}\,du \biggr) \,dA(s) +\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )} \,ds \Biggr]. \end{aligned}$$

In consequence, $\|\mathcal{F}_{1}(x,y)-\mathcal{F}_{1}(x,y)\|\rightarrow 0$ independent of x and y as $t_{2}\rightarrow t_{1}$. Also, we can obtain

$$\begin{aligned} &\bigl\vert \mathcal{F}_{2}(x,y) (t_{2})- \mathcal{F}_{2}(x,y) (t_{1}) \bigr\vert \\ &\quad\leq \frac{\theta _{2}}{\varGamma (\beta +1)} \bigl[2(t_{2}-t_{1})^{ \beta }+ \bigl\vert (t_{2}-a)^{\beta }-(t_{1}-a)^{\beta } \bigr\vert \bigr] \\ &\qquad{}+\theta _{1} \bigl\vert \psi _{1}(t_{2})- \psi _{1}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )} \,ds+\theta _{1} \bigl\vert \psi _{2}(t_{2})-\psi _{2}(t_{1}) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)} \,ds \\ &\qquad{}+\theta _{1} \bigl\vert \phi _{5}(t_{2})- \phi _{5}(t_{1}) \bigr\vert \Biggl[ \int _{a} ^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )} \,du \biggr) \,dA(s)+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )} \,ds \Biggr] \\ &\qquad{}+\theta _{2} \bigl\vert \psi _{4}(t_{2})- \psi _{4}(t_{1}) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \beta -1}}{\varGamma (\beta )} \,ds+\theta _{2} \bigl\vert \psi _{5}(t_{2})-\psi _{5}(t_{1}) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)} \,ds \\ &\qquad{}+\theta _{2} \bigl\vert \psi _{3}(t_{2})- \psi _{3}(t_{1}) \bigr\vert \Biggl[ \int _{a} ^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )}\,du \biggr) \,dA(s)+ \sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )} \,ds \Biggr], \end{aligned}$$

which imply that $\|\mathcal{F}_{2}(x,y)-\mathcal{F}_{2}(x,y)\|\rightarrow 0$ independent of x and y as $t_{2}\rightarrow t_{1}$. Therefore, the operator $\mathcal{F}(x,y)$ is equicontinuous. Then, by Arzelá–Ascoli theorem, the operator $\mathcal{F}$ is completely continuous.

Next, we show that the set $\mathcal{P}=\{(x,y)\in \mathcal{A}\times \mathcal{A}|(x,y)=\sigma \mathcal{F}(x,y), 0\leq \sigma \leq 1\}$ is bounded. Let $(x,y)\in \mathcal{P}$, then $(x,y)=\sigma \mathcal{F}(x,y)$ and for any $t\in [a,b]$, we have

$$ x(t)=\sigma \mathcal{F}_{1}(x,y) (t),\qquad y(t)=\sigma \mathcal{F}_{2}(x,y) (t). $$

In consequence, we have

$$\begin{aligned} \bigl\vert x(t) \bigr\vert \leq \varLambda _{1}\bigl( \varepsilon _{0}+\varepsilon _{1} \vert x \vert + \varepsilon _{2} \vert y \vert \bigr)+\varLambda _{2}\bigl(\kappa _{0}+\kappa _{1} \vert x \vert +\kappa _{2} \vert y \vert \bigr), \end{aligned}$$

which leads to

$$\begin{aligned} \Vert x \Vert \leq \varLambda _{1}\varepsilon _{0}+\varLambda _{2}\kappa _{0}+( \varLambda _{1}\varepsilon _{1}+\varLambda _{2}\kappa _{1}) \Vert x \Vert +(\varLambda _{1} \varepsilon _{2}+\varLambda _{2} \kappa _{2}) \Vert y \Vert . \end{aligned}$$

(3.11)

In a similar manner, we can find that

$$\begin{aligned} \Vert y \Vert \leq \varLambda _{3}\varepsilon _{0}+\varLambda _{4}\kappa _{0}+( \varLambda _{3}\varepsilon _{1}+\varLambda _{4}\kappa _{1}) \Vert x \Vert +(\varLambda _{3} \varepsilon _{2}+\varLambda _{4} \kappa _{2}) \Vert y \Vert . \end{aligned}$$

(3.12)

From (3.11) and (3.12) together with the notations (3.7) and (3.8), we get

$$\begin{aligned} \Vert x \Vert + \Vert y \Vert \leq{} & \bigl[(\varLambda _{1}+\varLambda _{3})\varepsilon _{0}+( \varLambda _{2}+\varLambda _{4})\kappa _{0} \bigr]+ \bigl[(\varLambda _{1}+\varLambda _{3}) \varepsilon _{1}+(\varLambda _{2}+\varLambda _{4})\kappa _{1} \bigr] \Vert x \Vert \\ &{}+ \bigl[(\varLambda _{1}+\varLambda _{3})\varepsilon _{2}+(\varLambda _{2}+\varLambda _{4}) \kappa _{2} \bigr] \Vert y \Vert . \end{aligned}$$

Thus,

$$\begin{aligned} \bigl\Vert (x,y) \bigr\Vert \leq \varOmega _{0}+\max \{ \varOmega _{1},\varOmega _{2}\} \bigl\Vert (x,y) \bigr\Vert \leq \varOmega _{0}+\varOmega \bigl\Vert (x,y) \bigr\Vert , \end{aligned}$$

which can alternatively be written as

$$ \bigl\Vert (x,y) \bigr\Vert \leq \frac{\varOmega _{0}}{1-\varOmega }. $$

This show that the set $\mathcal{P}$ is bounded. Therefore, by Lemma 3.1 (Leray–Schauder alternative theorem), the operator $\mathcal{F}$ has at least one fixed point, which implies that there exist at least one solution for the system (1.1)–(1.2) on $[a,b]$. □

Example 3.3

Consider the coupled system of fractional differential equations given by

$$ \textstyle\begin{cases} {} ^{c}D^{10/3}x(t)= \frac{9\sin (t)}{3+t^{3}} +\frac{14x(t) \vert y(t) \vert }{61(1+ \vert y(t) \vert )}+\frac{2 \sin y(t) \vert \tan ^{-1}x(t) \vert }{\pi \sqrt{4+t^{2}}}, \\ {} ^{c}D^{15/4}y(t)= \frac{2t}{15\sqrt{9+t^{4}}}+\frac{5\sin (x(t))}{8(t^{2}+2)}+ \frac{2y(t)}{\sqrt{121+2t ^{2}}}+\sin t,\quad t\in [0,1], \end{cases} $$

(3.13)

with the boundary conditions

$$ \textstyle\begin{cases} x'(0) = 0,\qquad x(1)=0,\qquad x'(1)=0,\\ x(0) = \int _{0}^{1}y(s)\,dA(s)+\sum_{i=1} ^{4}\alpha _{i} y(\xi _{i}) , \\ y'(0) = 0,\qquad y(1)=0,\qquad y'(1)=0,\\ y(0) = \int _{0}^{1}x(s)\,dA(s)+\sum_{i=1} ^{4}\beta _{i} x(\xi _{i}), \end{cases} $$

(3.14)

where $a=0, b=1, \alpha =\frac{10}{3}, \beta =\frac{15}{4}, \alpha _{1}=\frac{-1}{5}, \alpha _{2}=1, \alpha _{3}=\frac{3}{2}, \alpha _{4}=2, \beta _{1}=\frac{-1}{3}, \beta _{2}=0, \beta _{3}=\frac{1}{2}, \beta _{4}= \frac{5}{2}, \xi _{1}=\frac{1}{8}, \xi _{2}=\frac{1}{3}, \xi _{3}= \frac{1}{2}, \xi _{4}=\frac{2}{3}$, $f(t,x(t),y(t))= \frac{9\sin (t)}{3+t^{3}} +\frac{14x(t)|y(t)|}{61(1+|y(t)|)}+\frac{2 \sin y(t)|\tan ^{-1}x(t)|}{\pi \sqrt{4+t^{2}}}$, and $g(t,x(t),y(t))= \frac{2t}{15\sqrt{9+t^{4}}}+\frac{5\sin (x(t))}{8(t ^{2}+2)}+ \frac{2y(t)}{\sqrt{121+2t^{2}}}+\sin t$.

Let us take $A(s)= \frac{23(s^{2}+1)}{2}$. Using the given data, we have $A_{1}\simeq 15.8000, A _{2}\simeq 7.12188, A_{3}\simeq 5.41674, A_{4}\simeq 14.1667, A_{5} \simeq 6.98090, A_{6}\simeq 5.40259, \gamma _{1}\simeq 2.63395, \gamma _{2}\simeq -4.33906, \gamma _{3}\simeq 2.01460, \gamma _{4}\simeq -3.59290, \nu _{1}\simeq -2.64041, \nu _{2}\simeq -0.260460, \nu _{3}\simeq -0.520737, \nu _{4}\simeq 0.084305, \nu _{5}\simeq -0.049442, \nu _{6}\simeq 0.411084, \omega _{1}\simeq -0.501226, \omega _{2}\simeq 0.078035, \omega _{3} \simeq -0.199215, \omega _{4}\simeq -2.09815, \omega _{5}\simeq 0.339683, \delta _{1}\simeq -7.28082, \delta _{2}\simeq -0.520920, \delta _{3} \simeq -1.041470, \delta _{4}\simeq 0.168611, \delta _{5}\simeq -0.098885, \delta _{6}\simeq 1.82217, \lambda _{1}\simeq -1.00245, \lambda _{2} \simeq 0.156071, \lambda _{3}\simeq -0.398430, \lambda _{4}\simeq -6.19630, \lambda _{5}\simeq 1.67937, \mu _{1}\simeq 10.9212, \mu _{2}\simeq 0.781380, \mu _{3}\simeq 1.56220, \mu _{4}\simeq -0.252916, \mu _{5}\simeq 0.148328, \mu _{6}\simeq -2.23326, \rho _{1}\simeq 1.50368, \rho _{2}\simeq -0.234106, \rho _{3}\simeq 0.597645, \rho _{4}\simeq 9.29445, \rho _{5}\simeq -2.01906, \widetilde{\phi }_{1}\simeq 2.64041, \widetilde{\phi }_{2}\simeq 0.260460, \widetilde{\phi }_{3}\simeq 0.520737, \widetilde{\phi }_{4}\simeq 0.084305, \widetilde{\phi }_{5}\simeq 0.049442, \widetilde{\phi }_{6}\simeq 0.411084, \widetilde{\psi }_{1}\simeq 0.501226, \widetilde{\psi }_{2}\simeq 0.078035, \widetilde{\psi }_{3}\simeq 0.199215, \widetilde{\psi }_{4}\simeq 2.09815, \widetilde{\psi }_{5}\simeq 0.339683, \varLambda _{1}\simeq 0.681978, \varLambda _{2}\simeq 0.064064, \varLambda _{3}\simeq 0.108960, \varLambda _{4} \simeq 0.318420$.

Clearly,

$$\begin{aligned} &\bigl\vert f\bigl(t,x(t),y(t)\bigr) \bigr\vert \leq 3+\frac{14}{61} \Vert x \Vert +\frac{1}{2} \Vert y \Vert , \\ &\bigl\vert g\bigl(t,x(t),y(t)\bigr) \bigr\vert \leq \frac{2}{45}+ \frac{5}{16} \Vert x \Vert +\frac{2}{11} \Vert y \Vert , \end{aligned}$$

with $\varepsilon _{0}=3, \varepsilon _{1}=\frac{14}{61}, \varepsilon _{2}=\frac{1}{2}, \kappa _{0}=\frac{2}{45}, \kappa _{1}=\frac{5}{16}$, and $\kappa _{2}=\frac{2}{11}$. Using (3.8), we find that $\varOmega _{1}\simeq 0.301053, \varOmega _{2}\simeq 0.465012$ and $\varOmega = \max \{\varOmega _{1},\varOmega _{2}\}\simeq 0.465012<1$. Therefore, by Theorem 3.2, the problem (3.13)–(3.14) has at least one solution on $[0,1]$.

Our next result is based on the Krasnoselskii fixed point theorem.

Lemma 3.4

(Krasnoselskii)

Let $\mathcal{X}$be a closed, bounded, convex and nonempty subset of a Banach space $\mathcal{Y}$. Let $\mathcal{H}_{1}, \mathcal{H}_{2}$be operators mapping $\mathcal{X}$to $\mathcal{Y}$such that

• (a) $\mathcal{H}_{1}z_{1}+\mathcal{H}_{2}z_{2} \in \mathcal{X}$where $z_{1},z_{2} \in \mathcal{X}$;

• (b) $\mathcal{H}_{1}$is compact and continuous;

• (c) $\mathcal{H}_{2}$is a contraction mapping.

Then there exists $z \in \mathcal{X}$such that $z = \mathcal{H}_{1}z _{1}+\mathcal{H}_{2}z_{2}$.

Theorem 3.5

Assume that $f,g:[a,b]\times \mathbb{R}\times \mathbb{R}\rightarrow \mathbb{R}$are continuous functions satisfying the condition $(\mathcal{O}_{1})$. Furthermore, we assume that there exist two positive constants $B_{1}, B_{2}$such that $\forall t\in [a,b]$and $x,y\in \mathbb{R}$,

$$ \bigl\vert f(t,x,y) \bigr\vert \leq B_{1} \quad\textit{and}\quad \bigl\vert g(t,x,y) \bigr\vert \leq B_{2}. $$

(3.15)

Then system (1.1)–(1.2) has at least one solution on $[a,b]$, if

$$ (Q_{1}L_{1}+\varLambda _{2}L_{2})+(\varLambda _{3}L_{1}+Q_{2}L_{2})< 1, $$

(3.16)

where $Q_{1}= \varLambda _{1}-\frac{(b-a)^{\alpha }}{\varGamma (\alpha +1)}\textit{ and }Q _{2}= \varLambda _{4}-\frac{(b-a)^{\beta }}{\varGamma (\beta +1)}$.

Proof

Define a closed ball $S_{\eta }=\{(x,y)\in \mathcal{A}\times \mathcal{A}:\|(x,y)\|\leq \eta \}$ which is bounded and convex subset of the Banach space $\mathcal{A}\times \mathcal{A}$ and select

$$\begin{aligned} \eta \ge \max \{\varLambda _{1}B_{1}+ \varLambda _{2}B_{2}, \varLambda _{3}B_{1}+ \varLambda _{4}B_{2}\}. \end{aligned}$$

(3.17)

In order to verify the hypotheses of Lemma 3.4, we decompose the operator $\mathcal{F}$ into four operators $\mathcal{F}_{1,1}, \mathcal{F}_{1,2},\mathcal{F}_{2,1}$ and $\mathcal{F}_{2,2}$ on $S_{\eta }$ as follows:

$$\begin{aligned} \mathcal{F}_{1,1}(x,y) (t) ={}& \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{ \varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds, \\ \mathcal{F}_{1,2}(x,y) (t) ={}&{-}\phi _{1}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}-\phi _{6}(t) \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}f \bigl(s,x(s),y(s)\bigr)\,ds+ \phi _{2}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{ \varGamma (\alpha )} \\ &{}\times f\bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s) +\sum _{i=1}^{n-2}{\beta _{i}} \int _{a} ^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \Biggr] \\ &{}-\phi _{3}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}- \phi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}+\phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}g \bigl(u,x(u),y(u)\bigr)\,du \biggr)\,dA(s) \\ &{}+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \Biggr], \end{aligned}$$

and

$$\begin{aligned} \mathcal{F}_{2,1}(x,y) (t) ={}& \int _{a}^{t}\frac{(t-s)^{\beta -1}}{ \varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds, \\ \mathcal{F}_{2,2}(x,y) (t) ={}&{-}\psi _{1}(t) \int _{a}^{b}\frac{(b-s)^{ \alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}-\psi _{2}(t) \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}f \bigl(s,x(s),y(s)\bigr)\,ds +\phi _{5}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{ \varGamma (\alpha )} \\ &{}\times f\bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s)+\sum _{i=1}^{n-2}{\beta _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{\alpha -1}}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \Biggr] \\ &{}-\psi _{4}(t) \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}- \psi _{5}(t) \int _{a}^{b}\frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}g \bigl(s,x(s),y(s)\bigr)\,ds \\ &{}+\psi _{3}(t) \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{ \varGamma (\beta )}g \bigl(u,x(u),y(u)\bigr)\,du \biggr) \,dA(s) \\ &{}+\sum_{i=1}^{n-2}{\alpha _{i}} \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \Biggr]. \end{aligned}$$

Notice that $\mathcal{F}_{1}(x,y)(t)=\mathcal{F}_{1,1}(x,y)(t)+ \mathcal{F}_{1,2}(x,y)(t)$ and $\mathcal{F}_{2}(x,y)(t)=\mathcal{F} _{2,1}(x,y)(t)+\mathcal{F}_{2,2}(x,y)(t)$ on $S_{\eta }$. For verifying condition (a) of Lemma 3.4 we use (3.17) to show that $\mathcal{F}S_{\eta }\subset S_{\eta }$. Setting $x=(x_{1},x_{2}),y=(y _{1},y_{2}),\hat{x}=(\hat{x}_{1}, \hat{x}_{2})\text{ and }\hat{y}=( \hat{y}_{1}, \hat{y}_{2})\in S_{\eta }$, and using condition (3.15), we obtain

$$\begin{aligned} & \bigl\Vert \mathcal{F}_{1,1}(x,y) +\mathcal{F}_{1,2}( \hat{x}, \hat{y}) \bigr\Vert \\ &\quad\leq\sup_{t\in[a,b]}\Bigg\{ \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{\varGamma (\alpha )}B_{1} \,ds+ \bigl\vert \phi _{1}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\alpha -1}}{\varGamma (\alpha )}B _{1}\,ds \\ &\qquad{}+ \bigl\vert \phi _{6}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)}B_{1} \,ds+ \bigl\vert \phi _{2}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma ( \alpha )}B_{1} \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )}B_{1} \,ds \Biggr]+ \bigl\vert \phi _{3}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{ \beta -1}}{\varGamma (\beta )}B_{2} \,ds \\ &\qquad{}+ \bigl\vert \phi _{4}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)}B_{2} \,ds + \bigl\vert \phi _{5}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma ( \beta )}B_{2} \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )}B_{2} \,ds \Biggr]\Bigg\} \\ &\quad = \varLambda _{1}B_{1}+\varLambda _{2}B_{2}\le \eta. \end{aligned}$$

Similarly, we can find that

$$ \bigl\Vert \mathcal{F}_{2,1}(x,y) +\mathcal{F}_{2,2}( \hat{x},\hat{y}) \bigr\Vert \le \varLambda _{3}B_{1}+ \varLambda _{4}B_{2}\le \eta. $$

Clearly the above two inequalities lead to the fact that $\mathcal{F} _{1}(x,y)+\mathcal{F}_{2}(\hat{x},\hat{y})\in S_{\eta }$.

Now we prove that the operator $(\mathcal{F}_{1,2},\mathcal{F}_{2,2})$ is a contraction satisfying condition (c) of Lemma 3.4. For $(x_{1},y_{1}),(x_{2},y_{2})\in S_{\eta }$, we have

$$\begin{aligned} &\bigl\Vert \mathcal{F}_{1,2}(x_{1},y_{1})- \mathcal{F}_{1,2}(x_{2},y_{2}) \bigr\Vert \\ &\quad = \sup_{t\in [a,b]} \bigl\vert \mathcal{F}_{1,2} (x_{1},y_{1}) (t)- \mathcal{F}_{1,2} (x_{2},y_{2}) (t) \bigr\vert \\ &\quad\leq \sup_{t\in [a,b]} \Biggl\{ \bigl\vert \phi _{1}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x_{2}(s),y_{2}(s) \bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{6}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x _{2}(s),y_{2}(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{2}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(u,x_{1}(u),y_{1}(u)\bigr)-f\bigl(u,x_{2}(u),y_{2}(u) \bigr) \bigr\vert \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x_{2}(s),y _{2}(s)\bigr) \bigr\vert \,ds \Biggr] \\ &\qquad{}+ \bigl\vert \phi _{3}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x_{2}(s),y_{2}(s) \bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{4}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x _{2}(s),y_{2}(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{5}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(u,x_{1}(u),y_{1}(u)\bigr)-g\bigl(u,x_{2}(u),y_{2}(u) \bigr) \bigr\vert \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x_{2}(s),y _{2}(s)\bigr) \bigr\vert \,ds \Biggr] \Biggr\} \\ &\quad\leq \Biggl[ \Biggl(\widetilde{\phi }_{1} \frac{(b-a)^{\alpha }}{\varGamma (\alpha +1)}+\widetilde{\phi }_{6}\frac{(b-a)^{\alpha -1}}{\varGamma ( \alpha )}+ \widetilde{\phi }_{2} \Biggl( \int _{a}^{b}\frac{(s-a)^{\alpha }}{ \varGamma (\alpha +1)}\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert }\frac{(\xi _{i}-a)^{\alpha }}{\varGamma ( \alpha +1)} \Biggr) \Biggr)L_{1}+ \Biggl(\widetilde{\phi }_{3} \frac{(b-a)^{ \beta }}{\varGamma (\beta +1)}+\widetilde{\phi }_{4}\frac{(b-a)^{\beta -1}}{ \varGamma (\beta )} \\ &\qquad{}+ \widetilde{\phi }_{5} \Biggl( \int _{a}^{b}\frac{(s-a)^{ \beta }}{\varGamma (\beta +1)}\,dA(s) +\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert }\frac{(\xi _{i}-a)^{\beta }}{\varGamma ( \beta +1)} \Biggr) \Biggr)L_{2} \Biggr] \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr) \\ &\quad = (Q_{1}L_{1}+\varLambda _{2}L_{2} ) \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y _{2} \Vert \bigr) \end{aligned}$$

(3.18)

and

$$\begin{aligned} \bigl\Vert \mathcal{F}_{2,2}(x_{1},y_{1})- \mathcal{F}_{2,2}(x_{2},y_{2}) \bigr\Vert \le (\varLambda _{3}L_{1}+Q_{2}L_{2} ) \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y _{2} \Vert \bigr). \end{aligned}$$

(3.19)

It follows from (3.18) and (3.19) that

$$\begin{aligned} & \bigl\Vert (\mathcal{F}_{1,2}, \mathcal{F}_{2,2}) (x_{1}, y_{1})-(\mathcal{F} _{1,2}, \mathcal{F}_{2,2}) (x_{2},y_{2}) \bigr\Vert \\ &\quad\le \bigl[(Q_{1}L_{1}+\varLambda _{2}L_{2})+(\varLambda _{3}L_{1}+Q_{2}L_{2}) \bigr]\bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr), \end{aligned}$$

then by using (3.16) the operator $(\mathcal{F}_{1,2}, \mathcal{F}_{2,2})$ is a contraction. Therefore, the condition (c) of Lemma 3.4 is satisfied.

Next we will show that the operator $(\mathcal{F}_{1,1},\mathcal{F} _{2,1})$ satisfies the condition (b) of Lemma 3.4. By applying the continuity of the functions $f,g$ on $[a,b]\times \mathbb{R} \times \mathbb{R}$, we can conclude that the operator $(\mathcal{F} _{1,1},\mathcal{F}_{2,1})$ is continuous. For each $(x,y)\in S_{ \eta }$, we can have

$$\begin{aligned} \bigl\Vert \mathcal{F}_{1,1}(x,y) \bigr\Vert &\le \sup _{t\in [a,b]} \bigl\vert \mathcal{F}_{1,1}(x,y) (t) \bigr\vert \le \sup_{t\in [a,b]} \biggl\vert \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{\varGamma ( \alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ &\le \frac{(b-a)^{\alpha }B_{1}}{ \varGamma (\alpha +1)}=r_{1} \end{aligned}$$

and

$$\begin{aligned} \bigl\Vert \mathcal{F}_{2,1}(x,y) \bigr\Vert &\le \sup _{t\in [a,b]} \bigl\vert \mathcal{F}_{2,1}(x,y) (t) \bigr\vert \le \sup_{t\in [a,b]} \biggl\vert \int _{a}^{t}\frac{(t-s)^{\beta -1}}{\varGamma ( \beta )}g \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ & \le \frac{(b-a)^{\beta }B_{2}}{\varGamma (\beta +1)}=r_{2}, \end{aligned}$$

which yield

$$ \bigl\Vert (\mathcal{F}_{1,1},\mathcal{F}_{2,1}) (x,y) \bigr\Vert \leq r_{1}+r_{2}. $$

Thus the set $(\mathcal{F}_{1,1},\mathcal{F}_{2,1})S_{\eta }$ is uniformly bounded. In the next step, we will show that the set $(\mathcal{F}_{1,1},\mathcal{F}_{2,1})S_{\eta }$ is equicontinuous. For $t_{1},t_{2}\in [a,b]$ with $t_{1}< t_{2}$ and for any $(x,y)\in S_{ \eta }$, we obtain

$$\begin{aligned} \bigl\vert \mathcal{F}_{1,1}(x,y) (t_{2})- \mathcal{F}_{1,1}(x,y) (t_{1}) \bigr\vert \le {}& \biggl\vert \int _{a}^{t_{1}}\frac{[(t_{2}-s)^{\alpha -1}-(t_{1}-s)^{\alpha -1}]}{\varGamma (\alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ &{}+ \biggl\vert \int _{t_{1}}^{t_{2}}\frac{(t_{2}-s)^{\alpha -1}}{\varGamma ( \alpha )}f \bigl(s,x(s),y(s)\bigr)\,ds \biggr\vert \\ \le{} &\frac{B_{1}}{\varGamma (\alpha +1)} \bigl[2(t_{2}-t_{1})^{\alpha }+ \bigl\vert (t _{2}-a)^{\alpha }-(t_{1}-a)^{\alpha } \bigr\vert \bigr]. \end{aligned}$$

Analogously, we can get

$$\begin{aligned} \bigl\vert \mathcal{F}_{2,1}(x,y) (t_{2})- \mathcal{F}_{2,1}(x,y) (t_{1}) \bigr\vert \le \frac{B _{2}}{\varGamma (\beta +1)} \bigl[2(t_{2}-t_{1})^{\beta }+ \bigl\vert (t_{2}-a)^{ \beta }-(t_{1}-a)^{\beta } \bigr\vert \bigr]. \end{aligned}$$

Therefore, $|(\mathcal{F}_{1,1},\mathcal{F}_{2,1})(x,y)(t_{2})-( \mathcal{F}_{1,1},\mathcal{F}_{2,1})(x,y)(t_{1})|$ tends to zero as $t_{1}\rightarrow t_{2}$ independent of $(x,y)\in S_{\eta }$. Hence, the set $(\mathcal{F}_{1,1},\mathcal{F}_{2,1})S_{\eta }$ is equicontinuous. Thus it follows by the Arzelá–Ascoli theorem that the operator $(\mathcal{F}_{1,1},\mathcal{F}_{2,1})$ is compact on $S_{\eta }$. By the conclusion of Lemma 3.4, we deduce that system (1.1)–(1.2) has at least one solution on $[a,b]$. □

Example 3.6

Consider the same problem in Example 3.3 with the coupled boundary conditions (3.14) and

$$ f\bigl(t,x(t),y(t)\bigr)=\frac{2 \vert x(t) \vert }{3(1+ \vert x(t) \vert )}+\frac{2\cos y(t)}{\sqrt{t ^{2}+9}}+ \frac{t+2}{27},\quad t\in [0,1], $$

and

$$ g\bigl(t,x(t),y(t)\bigr)=\frac{\sin x(t)}{\sqrt{t^{4}+144}}+ \frac{y(t)+36}{12},\quad t\in [0,1]. $$

Clearly, $|f(t,x_{1},y_{1})-f(t,x_{2},y_{2})| \le \frac{2}{3} |x_{1}-x_{2}|+ \frac{2}{3}|y_{1}-y_{2}|$, $|g(t,x_{1},y_{1})-g(t,x_{2},y_{2})| \le \frac{1}{12} |x_{1}-x_{2}|+ \frac{1}{12}|y_{1}-y_{2}|$ and with the given data, we find that $Q_{1}\simeq 0.573992, Q_{2}\simeq 0.258129, \varLambda _{1}\simeq 0.681978, \varLambda _{2}\simeq 0.064064, \varLambda _{3}\simeq 0.108960, \varLambda _{4}\simeq 0.318420$. Note that and $(Q _{1}+\varLambda _{3})L_{1}+(Q_{2}+\varLambda _{2})L_{2} \simeq 0.482151<1$. Thus all the conditions of Theorem 3.5 are satisfied and consequently, its conclusion applies to the problem (3.13)–(3.14).

4 Uniqueness of solutions

This section is concerned with the uniqueness of solutions for the problem at hand and relies on the Banach contraction mapping principle.

Theorem 4.1

Assume that the condition $(\mathcal{O}_{1})$holds. Then system (1.1)–(1.2) has a unique solution on $[a,b]$if

$$ \Delta +\overline{\Delta }< 1, $$

(4.1)

where Δ and Δ̅ are given by (3.6).

Proof

Let us set $\tau > \frac{M+\overline{M}}{1-\Delta -\overline{\Delta }}$, where $\Delta ,\overline{\Delta },M$ and M̅ are given by (3.6), and show that $\mathcal{F}U_{\tau }\subset U_{\tau }$, where the operator $\mathcal{F}$ is given by (3.1) and

$$ U_{\tau }=\bigl\{ (x,y)\in \mathcal{A}\times \mathcal{A}: \bigl\Vert (x,y) \bigr\Vert \leq \tau \bigr\} . $$

In view of the assumption $(\mathcal{O}_{1})$ together with (3.5), for $(x,y)\in U_{\tau }, t\in [a,b]$, we have

$$\begin{aligned} \bigl\vert f\bigl(t,x(t),y(t)\bigr) \bigr\vert &\leq \bigl\vert f \bigl(t,x(t),y(t)\bigr)-f(t,0,0) \bigr\vert + \bigl\vert f(t,0,0) \bigr\vert \\ &\le L_{1} \bigl( \Vert x \Vert + \Vert y \Vert \bigr)+N_{1} \le L_{1} \tau + N_{1}, \\ \bigl\vert g\bigl(t,x(t),y(t)\bigr) \bigr\vert &\leq \bigl\vert g \bigl(t,x(t),y(t)\bigr)-g(t,0,0) \bigr\vert + \bigl\vert g(t,0,0) \bigr\vert \\ &\leq L_{2} \bigl( \Vert x \Vert + \Vert y \Vert \bigr)+N_{2} \leq L_{2} \tau + N_{2}. \end{aligned}$$

In view of (3.6), we obtain

$$\begin{aligned} \bigl\vert \mathcal{F}_{1}(x,y) (t) \bigr\vert &\leq \varLambda _{1}(L_{1} \tau + N_{1})+ \varLambda _{2}(L_{2} \tau + N_{2}) \\ &=( \varLambda _{1}L_{1}+\varLambda _{2}L_{2}) \tau +(\varLambda _{1}N_{1}+\varLambda _{2}N_{2}) =\Delta \tau +M, \\ \bigl\vert \mathcal{F}_{2}(x,y) (t) \bigr\vert & \leq \varLambda _{3}(L_{1} \tau + N_{1})+ \varLambda _{4}(L_{2} \tau + N_{2})\\ & =( \varLambda _{3}L_{1}+\varLambda _{4}L_{2}) \tau +(\varLambda _{3}N_{1}+\varLambda _{4}N_{2}) =\overline{\Delta }\tau + \overline{M}, \end{aligned}$$

which imply that

$$\begin{aligned} \bigl\Vert \mathcal{F}_{1}(x,y) \bigr\Vert \leq \Delta \tau + M, \qquad\bigl\Vert \mathcal{F}_{2}(x,y) \bigr\Vert \leq \overline{\Delta } \tau + \overline{M}. \end{aligned}$$

(4.2)

Thus, it follows from (4.2) that

$$\begin{aligned} \bigl\Vert \mathcal{F} (x,y) \bigr\Vert \leq (\Delta \tau +M)+(\overline{ \Delta }\tau + \overline{M}) \leq (\Delta +\overline{\Delta })\tau +(M+ \overline{M}) \leq \tau. \end{aligned}$$

Consequently, $\mathcal{F}U_{\tau } \subset U_{\tau }$. Next, we show that the operator $\mathcal{F}$ is a contraction. Using the conditions $(\mathcal{O}_{1})$, (3.6) and for any $(x_{1},y_{1}),(x_{2},y _{2})\in \mathcal{A}\times \mathcal{A}, t\in [a,b]$, we get

$$\begin{aligned} & \bigl\Vert \mathcal{F}_{1} (x_{1},y_{1})- \mathcal{F}_{1} (x_{2},y_{2}) \bigr\Vert \\ &\quad = \sup_{t\in [a,b]} \bigl\vert \mathcal{F}_{1} (x_{1},y_{1}) (t)- \mathcal{F}_{1} (x_{2},y_{2}) (t) \bigr\vert \\ &\quad\leq \sup_{t\in [a,b]} \Biggl\{ \int _{a}^{t}\frac{(t-s)^{\alpha -1}}{\varGamma ( \alpha )} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x_{2}(s),y_{2}(s) \bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{1}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x _{2}(s),y_{2}(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{6}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\alpha -2}}{\varGamma (\alpha -1)} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x _{2}(s),y_{2}(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{2}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(u,x_{1}(u),y_{1}(u)\bigr)-f\bigl(u,x_{2}(u),y_{2}(u) \bigr) \bigr\vert \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \beta _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \alpha -1}}{\varGamma (\alpha )} \bigl\vert f\bigl(s,x_{1}(s),y_{1}(s)\bigr)-f\bigl(s,x_{2}(s),y _{2}(s)\bigr) \bigr\vert \,ds \Biggr] \\ &\qquad{}+ \bigl\vert \phi _{3}(t) \bigr\vert \int _{a}^{b}\frac{(b-s)^{\beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x_{2}(s),y_{2}(s) \bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{4}(t) \bigr\vert \int _{a}^{b} \frac{(b-s)^{\beta -2}}{\varGamma (\beta -1)} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x _{2}(s),y_{2}(s)\bigr) \bigr\vert \,ds \\ &\qquad{}+ \bigl\vert \phi _{5}(t) \bigr\vert \Biggl[ \int _{a}^{b} \biggl( \int _{a}^{s}\frac{(s-u)^{\beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(u,x_{1}(u),y_{1}(u)\bigr)-g\bigl(u,x_{2}(u),y_{2}(u) \bigr) \bigr\vert \,du \biggr)\,dA(s) \\ &\qquad{}+\sum_{i=1}^{n-2}{ \vert \alpha _{i} \vert } \int _{a}^{\xi _{i}}\frac{(\xi _{i}-s)^{ \beta -1}}{\varGamma (\beta )} \bigl\vert g\bigl(s,x_{1}(s),y_{1}(s)\bigr)-g\bigl(s,x_{2}(s),y _{2}(s)\bigr) \bigr\vert \,ds \Biggr] \Biggr\} \\ &\quad\leq \varLambda _{1}L_{1} \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr)+ \varLambda _{2}L_{2} \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr) \\ &\quad = (\varLambda _{1}L_{1}+\varLambda _{2}L_{2} ) \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y _{1}-y_{2} \Vert \bigr) \\ &\quad =\Delta \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr). \end{aligned}$$

Similarly

$$\begin{aligned} \bigl\Vert \mathcal{F}_{2} (x_{1},y_{1})- \mathcal{F}_{2} (x_{2},y_{2}) \bigr\Vert &= \sup_{t\in [a,b]} \bigl\vert \mathcal{F}_{2} (x_{1},y_{1}) (t)- \mathcal{F}_{2} (x_{2},y_{2}) (t) \bigr\vert \\ &\leq (\varLambda _{3}L_{1}+\varLambda _{4}L_{2} ) \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr) \\ &=\overline{\Delta } \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr). \end{aligned}$$

Hence we obtain

$$ \bigl\| \mathcal{F} (x_{1},y_{1}) - \mathcal{F} (x_{2},y_{2}))\bigr\| \leq ( \Delta +\overline{\Delta }) \bigl( \Vert x_{1}-x_{2} \Vert + \Vert y_{1}-y_{2} \Vert \bigr), $$

which implies that $\mathcal{F}$ is a contraction by the assumption (4.1). Hence, we deduce by the conclusion of contraction mapping principle that there exists a unique solution for the problem (1.1)–(1.2) on $[a,b]$. □

Example 4.2

Consider the following system:

$$ \textstyle\begin{cases} {} ^{c}D^{10/3}x(t)= \frac{5}{2 \sqrt{81+t^{2}}} (\tan ^{-1}x(t)+\sin y(t) )+\frac{1}{25+t ^{2}}, \\ {} ^{c}D^{15/4}y(t)= \frac{t^{2}+16}{8 \sqrt{16+t^{6}}} (x(t)+\cos y(t) )+\frac{e ^{-t}}{3 \sqrt{t^{2}+9}},\quad t\in [0,1], \end{cases} $$

(4.3)

with the coupled boundary conditions (3.14).

Clearly, $|f(t,x_{1},y_{1})-f(t,x_{2},y_{2})|\leq L_{1}(\|x_{1}-x_{2} \|+\|y_{1}-y_{2}\|)$ with $L_{1}=\frac{5}{18}$ and $|g(t,x_{1},y_{1})-g(t,x _{2},y_{2})|\leq L_{2}(\|x_{1}-x_{2}\|+\|y_{1}-y_{2}\|)$ with $L_{2}=\frac{17}{32}$. Using the given data in Example (3.3) and (4.1), we find that $\Delta +\overline{\Delta }\simeq 0.422900<1$. Obviously, the hypothesis of Theorem 4.1 is satisfied. Hence, by the conclusion of Theorem 4.1 there is a unique solution for the problem (3.13) on $[0,1]$.