Fully coupled forward-backward stochastic differential equations on Markov chains

Abstract

We define fully coupled forward-backward stochastic differential equations on spaces related to continuous-time finite-state Markov chains. Existence and uniqueness results of the fully coupled forward-backward stochastic differential equations on Markov chains are obtained.

1 Introduction

Since the first introduction by Pardoux and Peng [1] in 1990, the theory of nonlinear backward stochastic differential equations (BSDEs) driven by a Brownian motion has been intensively researched by many researchers and has achieved abundant theoretical results. Now this theory is a powerful tool in stochastic analysis. It also has many important applications, namely in stochastic control, stochastic differential games, finance, and the theory of partial differential equations (PDEs).

In the classic BSDE theory, we consider a Brownian motion as the driver, but a Brownian motion is a kind of very idealized stochastic model, which limits greatly the applications of the classic BSDEs. There are many results about BSDEs associated with jump process. Tang and Li [2] first discussed BSDEs driven by a Brownian motion and Poisson process; Nualart and Schoutens [3] considered BSDEs driven by a Brownian motion and Lévy process. Furthermore, there are also results where the Brownian motion in the diffusion term of the BSDE is replaced by another process. For example, Cohen and Elliott [4] studied BSDEs driven by a continuous-time finite-state Markov chain. After that, many results, such as a comparison theorem about this kind of BSDEs, nonlinear expected results [5, 6], and so on, appeared.

Along with the rapid development of the BSDE theory, the theory of fully coupled forward-backward stochastic differential equations (FBSDEs), closely related to BSDEs, has been developed rapidly. Fully coupled FBSDEs with Brownian motion can be encountered in the optimization problem when applying stochastic maximum principle (see [7]) and in mathematical finance when considering a large investor in security market (see [8]). Such FBSDEs are also used in the potential theory (see [9]). As we know now, to get the existence and uniqueness results of fully coupled FBSDEs solutions, there are mainly three methods: the method of contraction mappings [10, 11], the four-step scheme [12], and the method of continuation [13, 14]. For more details on fully coupled FBSDEs, we refer to Yong [15], Ma et al. [16], or Briand and Hu [17] and the references therein.

In this paper, we study fully coupled FBSDEs driven by a martingale generated by a continuous-time finite-state Markov chain. Inspired by Peng and Wu [14], we introduce an $m\times n$ full-rank matrix G to overcome the problem caused by the different dimensions of SDE and BSDE. Using the method of continuation, the Itô product rule of semimartingales, and the fixed point principle, based on the theory of BSDEs driven by a continuous-time finite-state Markov chain, we obtain existence and uniqueness results of the FBSDEs on a Markov chain. It is worth pointing out that due to the property of the martingale generated by a finite-state Markov chain, the form of the monotone assumptions we employed here is different from that in Peng and Wu [14].

Recently, many works have been done on BSDEs with Markov chains. In fact, there are two major formulations of the state space of the Markov chain in the literature. One is the set of unit vectors in some Euclidean space [18]. The other one is the set of positive integers [19]. Each of the formulations was used by many researchers when they studied BSDEs driven by Markov chains. When a martingale representation was needed, the second formulation was necessary. The first one facilitates the mathematics, that is, the symbolism is nicer; the second one makes the mathematics more rigorous. Given a continuous-time Markov chain, assuming its state space be the set of positive integers, there exists an integer-valued random measure that counts the jumps of a Markov chain. Crepey and Matoussi [19] and Crepey [20] considered BSDEs driven by both a Brownian motion and the compensated martingale of a random measure. Tao et al. [21] discussed BSDEs with a singular perturbed Markov chain; Tao et al. [22, 23] considered the regime-switching system modulated by a Markov chain based on Crepey and Matoussi [19] and Crepey [20]. When the state space of a continuous-time Markov chain is described by the set of unit vectors in $R^{d}$, Zhang et al. [18] considered a system of Markov regime-switching model with Poisson jumps. Different from the above works, based on Cohen and Elliott [4], we discussed FBSDEs purely driven by a Markov chain rather than by a Brownian motion. Our model depends only on the state space of a Markov chain and leads to a special structure.

In Section 2, we formulate our problem; in Section 3, the preliminaries are given; in Section 4, we give our main result on the existence and uniqueness of the fully coupled FBSDE on a Markov chain and present the proofs; in Section 5, we discuss the existence and uniqueness result of the FBSDE under other monotone assumptions; in the last section, we give an application of these theoretical results to stochastic optimal control problem through an example.

2 Formulation of the problem

Consider a continuous-time finite-state Markov chain $m=\{m_{t},t\in [0,T]\}$. Following the convention of Elliott et al. [24], we assume that it takes values in unit vectors $e_{i}$ in $R^{d}$, where d is the number of states of the chain.

We consider stochastic processes defined on the filtered probability space $(\Omega,\mathscr{F},\mathscr{F}_{t},\mathbb{P})$, where $\{\mathscr{F}_{t}\}$ is the completed natural filtration generated by the σ-fields $\mathscr{F}_{t}=\sigma(\{m_{s}, s\leq t \},F\in\mathscr{F}_{T}:\mathbb{P}(F)=0)$, and $\mathscr{F}=\mathscr{F}_{T}$. Note that if m is right-continuous, then this filtration is right-continuous. If $\mathcal{A}_{t}$ denotes the rate matrix for m at time t, then this chain has the following representation:

$$m_{t}=m_{0}+ \int_{0}^{t}\mathcal {A}_{s}m_{s}\,ds+M_{t}, $$

where $M_{t}$ is a martingale (see [24]).

We consider the following forward-backward stochastic differential equations:

$$ \left \{ \textstyle\begin{array}{@{}l} X_{t} = x + \int_{0}^{t}b(s,X_{s},Y_{s},Z_{s})\,ds+ \int_{0}^{t}\sigma(s-,X_{s-},Y_{s-},Z_{s-})\,dM_{s}, \\ Y_{t} = \Phi(X_{T}) + \int_{t}^{T}f(s,X_{s},Y_{s},Z_{s})\,ds- \int_{t}^{T}Z_{s-}\,dM_{s}, \end{array}\displaystyle \right . $$

(1)

where X, Y, Z take values in $\mathbb{R}^{n}$, $\mathbb{R}^{m}$, $\mathbb{R}^{m\times d}$, $T>0$ is an arbitrary fixed number, and b, σ, f, Φ are functions of appropriate dimensions:

$$\begin{aligned}& b: \Omega\times[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{m} \times \mathbb{R}^{m\times d}\longrightarrow\mathbb{R}^{n}, \\& \sigma: \Omega\times[0,T]\times\mathbb{R}^{n}\times\mathbb {R}^{m}\times\mathbb{R}^{m\times d}\longrightarrow \mathbb{R}^{n\times d}, \\& f: \Omega\times[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{m} \times \mathbb{R}^{m\times d}\longrightarrow\mathbb{R}^{m}, \\& \Phi: \Omega\times\mathbb{R}^{n}\longrightarrow\mathbb{R}^{m}. \end{aligned}$$

We shall seek an $\mathscr{F}_{t}$-adapted triple $(X,Y,Z)$ that satisfies the forward-backward stochastic differential equation (1) on $[0,T]$ $\mathbb{P}$-almost surely. That is, our aim is to find an $\mathscr{F}_{t}$-adapted solution of (1).

Noting that only the s-left limit $\sigma(\omega, s-,X_{s-},Y_{s-},Z_{s-})$ enters into FBSDE (1), without loss of generality, we assume that $\sigma (s,X_{s},Y_{s},Z_{s})$ is left-continuous in s for all w and X, Y. Since M is a semimartingale, X, Y is càdlàg and adapted. We suppose the existence of the left limits of Z. So Z must have at most a countable number of discontinuities, and then it must be left-continuous at each t except possibly on a dt-null set. Hence, if $Z_{s}$ satisfies FBSDE (1), then so does $Z_{s-}$:

$$ \left \{ \textstyle\begin{array}{@{}l} X_{t} = x + \int_{0}^{t}b(s,X_{s},Y_{s},Z_{s-})\,ds+ \int_{0}^{t}\sigma(s-,X_{s-},Y_{s-},Z_{s-})\,dM_{s}, \\ Y_{t}= \Phi(X_{T}) + \int_{t}^{T}f(s,X_{s},Y_{s},Z_{s-})\,ds- \int_{t}^{T}Z_{s-}\,dM_{s}. \end{array}\displaystyle \right . $$

Setting $Z_{t}^{**}:=Z_{t-}$, we have a left-continuous process $Z^{**}$, which also satisfies the desired equations. Therefore, writing the left limits $Z_{t-}$ is unnecessary since we simply assume that our solution is left-continuous.

Based on these arguments, we rewrite FBSDE (1) as

$$ \left \{ \textstyle\begin{array}{@{}l} X_{t} = x + \int_{0}^{t}b(s,X_{s},Y_{s},Z_{s})\,ds+ \int_{0}^{t}\sigma(s,X_{s-},Y_{s-},Z_{s})\,dM_{s}, \\ Y_{t} = \Phi(X_{T}) + \int_{t}^{T}f(s,X_{s},Y_{s},Z_{s})\,ds- \int_{t}^{T}Z_{s}\,dM_{s}. \end{array}\displaystyle \right . $$

Recall that FBSDE (1) is equivalent to

$$ \left \{ \textstyle\begin{array}{@{}l} X_{t} = x + \int_{0}^{t}b^{**}(s,X_{s},Y_{s},Z_{s})\,ds+ \int_{0}^{t}\sigma(s,X_{s-},Y_{s-},Z_{s})\,dm_{s}, \\ Y_{t} = \Phi(X_{T}) + \int_{t}^{T}f^{**}(s,X_{s},Y_{s},Z_{s})\,ds- \int_{t}^{T}Z_{s}\,dm_{s}, \end{array}\displaystyle \right . $$

(2)

where $b^{**}(s,X_{s},Y_{s},Z_{s})=b(s,X_{s},Y_{s},Z_{s})-\sigma (s,X_{s},Y_{s},Z_{s})\mathcal {A}_{s}m_{s}$ and $f^{**}(s,X_{s},Y_{s},Z_{s})=f(s,X_{s}, Y_{s},Z_{s})+Z_{s}\mathcal {A}_{s}m_{s}$.

Obviously, FBSDE (2) is driven by the Markov chain $m=\{ m_{t},t\in[0,T]\}$.

3 Preliminaries

3.1 Preliminary notation

Elliott et al. [24] obtained the optional quadratic variation of $M_{t}$ as the matrix process

$$[M,M]_{t}=\sum_{s=0}^{t} \bigtriangleup M_{s} \triangle M_{s}^{*}, $$

where ‘^∗’ means transpose.

Observing that $\mathcal{A}$ is the rate matrix of the Markov chain m, the predictable quadratic variation is

$$\langle M,M\rangle_{t}= \int_{0}^{t}\bigl[\operatorname{diag}(\mathcal {A}_{s}m_{s})-\operatorname{diag}(m_{s}) \mathcal{A}_{s}^{*}-\mathcal{A}_{s} \operatorname{diag}(m_{s})\bigr]\,ds. $$

This can be seen through considering

$$\operatorname{diag}(m_{t})=\operatorname{diag}(m_{0})+ \int_{0}^{t}\operatorname{diag}( \mathcal{A}_{s}m_{s})\,ds+ \int _{0}^{t}\operatorname{diag}(dM_{s}) $$

and

$$\begin{aligned} \operatorname{diag}(m_{t})={}&m_{t}m_{t}^{*}\\ ={}&m_{0}m_{0}^{*}+ \int_{0}^{t}m_{s}m_{s}^{*} \mathcal {A}_{s}^{*}\,ds+ \int_{0}^{t}m_{s-}\,dM_{s}^{*}+ \int_{0}^{t}\mathcal {A}_{s}m_{s}m_{s}^{*}\,ds\\ &{}+ \int_{0}^{t}\,dM_{s}m_{s-}^{*}+ \sum_{s=0}^{t} \bigtriangleup M_{s} \triangle M_{s}^{*}. \end{aligned}$$

Equating these two, we get

$$[M,M]_{t}=L_{t}+ \int_{0}^{t}\bigl[\operatorname{diag}(\mathcal {A}_{s}m_{s})-\operatorname{diag}(m_{s}) \mathcal{A}_{s}^{*}-\mathcal {A}_{s} \operatorname{diag}(m_{s})\bigr]\,ds, $$

where L is some martingale. This in turn suggests that

$$\langle M,M\rangle_{t}= \int_{0}^{t}\bigl[\operatorname{diag}(\mathcal {A}_{s}m_{s})-\operatorname{diag}(m_{s}) \mathcal{A}_{s}^{*}-\mathcal {A}_{s} \operatorname{diag}(m_{s})\bigr]\,ds. $$

We will use the following notation:

By $(\cdot,\cdot)$ we denote the usual inner product in $\mathbb{R}^{n}$ or $\mathbb{R}^{m}$; we use the usual Euclidean norm in $\mathbb{R}^{n}$ and $\mathbb{R}^{m}$; and for $z\in\mathbb {R}^{m\times d}$, we define $|z|=\{\operatorname{tr}(zz^{*})\}^{\frac{1}{2}}$.

For $z^{1}\in\mathbb{R}^{m\times d}$, $z^{2}\in\mathbb{R}^{m\times d}$,

$$\bigl(\bigl(z^{1}, z^{2}\bigr)\bigr)=\operatorname{tr} \bigl(z^{1}\bigl(z^{2}\bigr)^{*}\bigr), $$

and for $u^{1}=(x^{1},y^{1},z^{1})\in\mathbb{R}^{n}\times\mathbb {R}^{m}\times\mathbb{R}^{m\times d}$, $u^{2}=(x^{2},y^{2},z^{2})\in\mathbb{R}^{n}\times\mathbb {R}^{m}\times\mathbb{R}^{m\times d}$,

$$\bigl[u^{1},u^{2}\bigr]=\bigl(x^{1}, x^{2}\bigr)+\bigl(y^{1}, y^{2} \bigr)+\bigl( \bigl(z^{1}, z^{2}\bigr)\bigr). $$

We are given an $m\times n$ full-rank matrix G. For $u=(x,y,z)\in\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d}$, let

$$\begin{aligned}& F(t,u)=\bigl(-G^{*}f(t,u), Gb(t,u), 0\bigr), \qquad H(t,u)=\bigl(0, 0, G\sigma(t,u)\bigr), \end{aligned}$$

where $G\sigma=(G\sigma_{1} \cdots G\sigma_{d})$.

Furthermore, we define

$$\begin{aligned}& \langle C,D\rangle_{V}=\operatorname{tr}\bigl(C\bigl[\operatorname{diag}(\mathcal {A}_{s}v)-\operatorname{diag}(v)\mathcal{A}_{s}^{*}- \mathcal {A}_{s}\operatorname{diag}(v)\bigr]D^{*}\bigr),\qquad \|C\|_{v}^{2}=\langle C,C\rangle_{v}, \end{aligned}$$

where v is a basis vector in $\mathbb{R}^{d}$.

3.2 BSDEs on Markov chains

We recall the existence and uniqueness results for a solution to the following BSDE on a Markov chain:

$$Y_{t} = \Phi(X_{T}) - \int_{t}^{T}F_{1}(\omega,s,Y_{s},Z_{s})\,ds- \int_{t}^{T}\bigl[F_{2}(s,Y_{s-})+Z_{s} \bigr]\,dM_{s} $$

for functions $F_{1}:\Omega\times[0,T]\times\mathbb{R}^{n}\times \mathbb{R}^{n \times d}$ and $F_{2}:\Omega\times[0,T]\times \mathbb{R}^{n}$; these functions are assumed to be progressively measurable, that is, $F_{1}(\cdot,s,Y_{s},Z_{s})$ and $F_{2}(\cdot,s,Y_{s})$ are $\mathscr{F}_{t}$-measurable for all $t\in[0,T]$.

Cohen and Elliott [4] gave the following result about martingale representation.

Lemma 1

Any $R^{d}$-valued martingale L defined on $(\Omega,\mathscr{F}_{t},\mathbb{P})$ can be represented as a stochastic (in this case, Stieltjes) integral with respect to the martingale process M up to equality $\mathbb{P}$-a.s. This representation is unique up to a $d\langle M,M \rangle_{t}\times \mathbb{P}$-null set. That is,

$$L_{t}=L_{0}+ \int_{0}^{t}Z_{s}\,dM_{s}, $$

where $Z_{s}$ is a predictable $\mathbb{R}^{n\times d}$-valued matrix process.

Cohen and Elliott [4] also obtained the existence and uniqueness of a solution to the above BSDE.

Theorem 1

Assume the Lipschitz continuity on the generators $F_{2}$ and $F_{2}$, that is, that there exists $c\in\mathbb{R}$ such that, for all $s\in[0,T]$,

$$\begin{aligned}& E\bigl|F_{1}\bigl(s,Y_{s}^{1},Z_{s}^{1} \bigr)-F_{1}\bigl(s,Y_{s}^{2},Z_{s}^{2} \bigr)\bigr|^{2}\leq c^{2}E\bigl|Y_{s}^{1}-Y_{s}^{2}\bigr|^{2}+c^{2}E \bigl\| Z_{s}^{1}-Z_{s}^{2} \bigr\| _{M_{s}}^{2},\\& E\bigl\| F_{2}\bigl(s,Y_{s-}^{1}\bigr)-F_{2} \bigl(s,Y_{s-}^{2}\bigr)\bigr\| _{M_{s}}^{2}\leq c^{2}E\bigl|Y_{s}^{1}-Y_{s}^{2}\bigr|^{2}. \end{aligned}$$

Under this Lipschitz condition, the above equation has at most one solution up to indistinguishability for Y and equality $d\langle M,M\rangle_{t}\times\mathbb{P}$-a.s. for Y.

4 Existence and uniqueness for FBSDE (1)

We give two definitions.

Definition 1

We denote by $\mathcal {M}^{2}(0,T;\mathbb{R}^{n})$ the set of all ${R}^{n}$-valued $\mathscr{F}_{t}$-adapted processes such that

$$E \int_{0}^{T}\bigl|v(s)\bigr|^{2}\,ds< +\infty. $$

Definition 2

A triple of processes $(X,Y,Z): \Omega\times [0,T] \rightarrow\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d}$ is called an adapted solution of the FBSDE (1) if $(X,Y,Z)\in\mathcal {M}^{2}(0,T;\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d})$ and it satisfies FBSDE (1) $\mathbb{P}$-almost surely.

The adaptedness of the solution enables us to rewrite FBSDE (1) in a differential form:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t} = b(t,X_{t},Y_{t},Z_{t})\,dt+\sigma (t,X_{t-},Y_{t-},Z_{t})\,dM_{t}, \\ -dY_{t} = f(t,X_{t},Y_{t},Z_{t})\,dt-Z_{t}\,dM_{t}, \\ X_{0} =x,\qquad Y_{T}=\Phi(X_{T}). \end{array}\displaystyle \right . $$

Now we give the main assumptions of our paper.

Assumption 1

For each $u=(x,y,z)\in\mathbb{R}^{n}\times \mathbb{R}^{m}\times\mathbb{R}^{m\times d}$, $F(\cdot,u), H(\cdot ,u)\in\mathcal{M}^{2}(0,T;\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d})$, and for each $x\in\mathbb{R}^{n}$, $\Phi(x)\in L^{2}(\Omega,\mathscr{F}_{T};\mathbb{R}^{n})$. Moreover,

• (i) $F(t,u)$ is uniformly Lipschitz with respect to u;

• (ii) $H(t,u)$ is uniformly Lipschitz with respect to u;

• (iii) $\Phi(x)$ is uniformly Lipschitz with respect to x.

Assumption 2

There exists constants $c_{2}$, $c_{2}^{\prime}$, $c_{3}$ such that

$$\begin{aligned}& \bigl[F\bigl(t,u^{1}\bigr)-F\bigl(t,u^{2}\bigr), u^{1}-u^{2}\bigr] \leq -c_{2}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2}-c_{2}^{\prime} \bigl(\bigl|G^{*}\bigl(y^{1}-y^{2} \bigr)\bigr|^{2}+\bigl|G^{*}\bigl(z^{1}-z^{2} \bigr)\bigr|^{2}\bigr),\\& \bigl[H\bigl(t,u^{1}\bigr)-H\bigl(t,u^{2}\bigr), u^{1}-u^{2}\bigr] \leq -c_{2}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2}-c_{2}^{\prime} \bigl(\bigl|G^{*}\bigl(y^{1}-y^{2} \bigr)\bigr|^{2}+\bigl|G^{*}\bigl(z^{1}-z^{2} \bigr)\bigr|^{2}\bigr),\\& \quad\forall u^{1}=\bigl(x^{1},y^{1},z^{1} \bigr), u^{2}=\bigl(x^{2},y^{2},z^{2} \bigr) \in\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d},\ \mathbb{P}\mbox{-a.s., a.e. }t\in\mathbb{R}^{+}, \end{aligned}$$

and

$$\bigl(\Phi\bigl(x^{1}\bigr)-\Phi\bigl(x^{2}\bigr), G \bigl(x^{1}-x^{2}\bigr)\bigr)\geq c_{3}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2},\quad \forall x^{1}\in\mathbb{R}^{n}, \forall x^{2}\in \mathbb{R}^{n}, $$

where $c_{2}$, $c_{2}^{\prime}$, and $c_{3}$ are given positive constants.

Now we give the main result of our paper.

Theorem 2

Let Assumptions 1 and 2 hold. Then there exists a unique adapted solution $(X,Y,Z)$ of FBSDE (1).

Proof of uniqueness

Let $U^{1}=(X^{1},Y^{1},Z^{1})$ and $U^{2}=(X^{2},Y^{2},Z^{2})$ be two adapted solutions of FBSDE (1). We set

$$\begin{aligned}& (\hat{X},\hat{Y},\hat{Z})=\bigl(X^{1}-X^{2},Y^{1}-Y^{2},Z^{1}-Z^{2} \bigr),\\& \left \{ \textstyle\begin{array}{@{}l} \hat{b}(t) = b(t,U_{t}^{1})-b(t,U_{t}^{2}), \\ \hat{\sigma}(t) = \sigma(t,U_{t}^{1})-\sigma(t,U_{t}^{2}), \\ \hat{f}(t) =f(t,U_{t}^{1})-f(t,U_{t}^{2}). \end{array}\displaystyle \right . \end{aligned}$$

Then we have

$$ \left \{ \textstyle\begin{array}{@{}l} d\hat{X_{t}} = \hat{b}(t,X_{t},Y_{t},Z_{t})\,dt+\hat{\sigma }(t,X_{t-},Y_{t-},Z_{t})\,dM_{t}, \\ -d\hat{Y_{t}} = \hat{f}(t,X_{t},Y_{t},Z_{t})\,dt-\hat{Z_{t}}\,dM_{t}. \end{array}\displaystyle \right . $$

From Assumption 1 it follows that

$$E\bigl({\sup}_{t\in[0,T]}|\hat{X}_{t}|^{2}\bigr)+E \bigl({\sup}_{t\in[0,T]}|\hat {Y}_{t}|^{2}\bigr)< +\infty. $$

Using the Itô product rule of semimartingales, we have

$$d(G\hat{X_{s}}, \hat{Y_{s}})=\hat{Y_{s-}}d(G \hat{X_{s}})+(G\hat {X_{s-}})\,d\hat{Y_{s}}+d(G \hat{X_{s}})\,d\hat{Y_{s}}, $$

where $d(G\hat{X_{s}})\,d\hat{Y_{s}}=d[G\hat{X_{s}},\hat{Y_{s}}]_{t}$, and hence, taking expectation and evaluating at $t=T$, by Assumption 2 we get

$$\begin{aligned} E(\hat{Y_{T}},G\hat{X_{T}}) =&E\bigl(\Phi \bigl(X_{T}^{1}\bigr)-\Phi \bigl(X_{T}^{2} \bigr),G\bigl(X_{T}^{1}-X_{T}^{2}\bigr) \bigr) \\ =&E \int_{0}^{T}\bigl[\bigl(-G^{*}\hat{f}, \hat{X_{s}}\bigr)+(G\hat{b},\hat {Y_{s}})\bigr]\,ds+E\sum _{s=0}^{T}\bigl((G\hat{\sigma}, \hat{Z_{s}})\bigr)\triangle M_{s}\triangle M_{s}^{*} \\ =&E \int_{0}^{T}\bigl[F\bigl(s,U^{1}\bigr)-F \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,ds \\ &{} +E\sum_{s=0}^{T}\bigl[H \bigl(s,U^{1}\bigr)-H\bigl(s,U^{2}\bigr),U^{1}-U^{2} \bigr] \triangle M_{s}\triangle M_{s}^{*} \\ =&E \int_{0}^{T}\bigl[F\bigl(s,U^{1}\bigr)-F \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,ds \\ &{} +E \int_{0}^{T}\bigl[H\bigl(s,U^{1}\bigr)-H \bigl(s,U^{2}\bigr),U^{1}-U^{2} \bigr]\,d[M,M]_{s} \\ =&E \int_{0}^{T}\bigl[F\bigl(s,U^{1}\bigr)-F \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,ds \\ &{} +E \int_{0}^{T}\bigl[H\bigl(s,U^{1}\bigr)-H \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,d \langle M,M \rangle_{s} \\ \leq&-c_{2}E \int_{0}^{T}|G\hat{X}|^{2}\,ds-c_{2}^{\prime}E \int _{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,ds \\ &{} -c_{2}E \int_{0}^{T}|G\hat{X}|^{2}\,d\langle M,M \rangle _{s}-c_{2}^{\prime}E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,d\langle M,M \rangle_{s}. \end{aligned}$$

Then we deduce that

$$\begin{aligned} &c_{2}\biggl(E \int_{0}^{T}|G\hat{X}|^{2}\,ds+E \int _{0}^{T}|G\hat{X}|^{2}\,d\langle M,M \rangle_{s}\biggr) \\ &\qquad{} +c_{2}^{\prime}\biggl[E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat {Z}\bigr|^{2}\bigr)\,ds+E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,d\langle M,M \rangle_{s}\biggr]+c_{3}|G\hat{X}|^{2} \\ &\quad \leq0. \end{aligned}$$

Since $c_{2}$, $c_{2}^{\prime}$, and $c_{3}$ are given positive constants, we have $|G\hat{X}|^{2}\equiv0$, $|G^{*}\hat{Y}|^{2}\equiv0$, $|G^{*}\hat {Z}|^{2}\equiv0$. So $\hat{X}_{s}^{1}\equiv\hat{X}_{s}^{2}$, $\hat {Y}_{s}^{1}\equiv\hat{Y}_{s}^{2}$, $\hat{Z}_{s}^{1}\equiv\hat{Z}_{s}^{2}$, $d \langle M,M \rangle_{t}\times P$-a.s. □

We now consider the following family of FBSDEs parameterized by $l\in[0,1]$:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{l} =[(1-l)c_{2}^{\prime}(-G^{*}Y_{t}^{l})+lb(t, U_{t}^{l})+\phi _{t}]\,dt\\ \hphantom{dX_{t}^{l} =}{}+[(1-l)c_{2}^{\prime}(-G^{*}Z_{t}^{l})+l\sigma(t, U_{t}^{l})+\psi _{t}]\,dM_{t}, \\ -dY_{t}^{l} =[(1-l)c_{2}GX_{t}^{l}+lf(t, U_{t}^{l})+\gamma _{t}]\,dt-Z_{t}^{l}\,dM_{t}, \\ X_{0}^{l} =x,\qquad Y_{T}^{l}=l\Phi(X_{T}^{l})+(1-l)GX_{T}^{l}+\xi, \end{array}\displaystyle \right . $$

(3)

where ϕ, ψ, and γ are given processes in $M^{2}(0,T)$ with values in $R^{n}$, $R^{n\times d}$, and $R^{m}$, respectively. Clearly, when $l=1$, $\xi\equiv0$, the existence of a solution of FBSDE (3) implies the existence of that of FBSDE (1). When $l=0$ in FBSDE (3), it is

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{0}=[-c_{2}^{\prime}G^{*}Y_{t}^{0}+\phi _{t}]\,dt+[-c_{2}^{\prime}G^{*}Z_{t}^{0}+\psi_{t}]\,dM_{t}, \\ -dY_{t}^{0} =[c_{2}GX_{t}^{0}+\gamma_{t}]\,dt-Z_{t}^{0}\,dM_{t}, \\ X_{0}^{l} =x,\qquad Y_{T}^{0}=GX_{T}^{0}+\xi. \end{array}\displaystyle \right . $$

(4)

For proving the existence part of the theorem, we first need the following two lemmas.

Lemma 2

The following equation has a unique solution:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t} =[-c_{2}^{\prime}G^{*}Y_{t}+\phi_{t}]\,dt+[-c_{2}^{\prime}G^{*}Z_{t}+\psi _{t}]\,dM_{t}, \\ -dY_{t} =[c_{2}GX_{t}+\gamma_{t}]\,dt-Z_{t}\,dM_{t}, \\ X_{0} =x,\qquad Y_{T}=\lambda GX_{T}+\xi, \end{array}\displaystyle \right . $$

(5)

where λ is a nonnegative constant.

Proof

We note that the matrix G is of full rank. The proof of the existence for FBSDE (5) can be divided into two cases, $n\leq m$ and $n>m$.

For the first case, the matrix $G^{*}G$ is of full rank. Let

$$ \left ( \textstyle\begin{array}{@{}c@{}} X^{\prime} \\ Y^{\prime} \\ Z^{\prime} \end{array}\displaystyle \right ) = \left ( \textstyle\begin{array}{@{}c@{}} X \\ G^{*}Y \\ G^{*}Z \end{array}\displaystyle \right ), \qquad \left ( \textstyle\begin{array}{@{}c@{}} Y^{\prime\prime} \\ Z^{\prime\prime} \end{array}\displaystyle \right )=\left ( \textstyle\begin{array}{@{}c@{}} (I_{m}-G(G^{*}G)^{-1}G^{*})Y \\ (I_{m}-G(G^{*}G)^{-1}G^{*})Z \end{array}\displaystyle \right ). $$

Multiplying $G^{*}$ for $(Y,Z)$ on both sides of the BSDE yields

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{\prime} =[-c_{2}^{\prime}Y_{t}^{\prime}+\phi_{t}]\,dt+[-c_{2}^{\prime}Z_{t}^{\prime}+\psi _{t}]\,dM_{t}, \\ -dY_{t}^{\prime} =[c_{2}G^{*}GX_{t}^{\prime}+G^{*}\gamma _{t}]\,dt-Z_{t}^{\prime}\,dM_{t}, \\ X_{0}^{\prime} =x,\qquad Y_{T}^{\prime}=\lambda G^{*}GX_{T}^{\prime}+G^{*}\xi. \end{array}\displaystyle \right . $$

(6)

Similarly, multiplying $(I_{m}-G(G^{*}G)^{-1}G^{*})$ on both sides of the same equation gives

$$ \left \{ \textstyle\begin{array}{@{}l} -dY_{t}^{\prime\prime} =(I_{m}-G(G^{*}G)^{-1}G^{*})\gamma _{t}\,dt-Z_{t}^{\prime\prime}\,dM_{t}, \\ Y_{T}^{\prime\prime} =(I_{m}-G(G^{*}G)^{-1}G^{*})\xi. \end{array}\displaystyle \right . $$

Obviously, the pair $(Y^{\prime\prime},Z^{\prime\prime})$ is uniquely determined (see Cohen and Elliott [4]). The uniqueness of $(X^{\prime},Y^{\prime},Z^{\prime})$ follows from Theorem 2. In order to solve FBSDE (6), we introduce the following $n\times n$-symmetric matrix-valued ODE, which is known as the matrix-Riccati equation:

$$ \left \{ \textstyle\begin{array}{@{}l} -\dot{K}(t) =-c_{2}^{\prime}K^{2}+c_{2}G^{*}G,\quad t\in[0,T], \\ K_{T} =\lambda G^{*}G. \end{array}\displaystyle \right . $$

It is well known that this equation has a unique nonnegative solution $K \in C^{1}([0,T);S^{n})$, where $S^{n}$ stands for the space of all $n\times n$-symmetric matrices. We then study the solution $(p,q) \in M^{2}(0,T;R^{n+n\times d})$ of the following linear simple BSDE:

$$ \left \{ \textstyle\begin{array}{@{}l} -dp_{t} =[-c_{2}^{\prime}K(t)p_{t}+K(t)\phi_{t}+G^{*}\gamma _{t}]\,dt+[K(t)\psi_{t}-(I_{n}+c_{2}^{\prime}K(t))q_{t}]\,dM_{t},\quad t\in[0,T],\\ p_{T} =G^{*}\xi. \end{array}\displaystyle \right . $$

We now consider $X_{t}^{\prime}$ as the solution of the SDE

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{\prime} =[-c_{2}^{\prime}(K(t)X_{t}^{\prime}+p_{t})+\phi_{t}]\,dt+[\psi _{t}-c_{2}^{\prime}q_{t}]\,dM_{t}, \\ X_{0}^{\prime} =x. \end{array}\displaystyle \right . $$

Then we can easily check that $(X_{t}^{\prime},Y_{t}^{\prime},Z_{t}^{\prime})=(X_{t}^{\prime},K(t)X_{t}^{\prime}+p_{t},q_{t})$ is a solution of FBSDE (6). Once $(X^{\prime},Y^{\prime},Z^{\prime})$ and $(Y^{\prime\prime},Z^{\prime\prime})$ are resolved, we can obtain the triple $(X,Y,Z)$ uniquely by

$$ \left ( \textstyle\begin{array}{@{}c@{}} X \\ Y \\ Z \end{array}\displaystyle \right ) = \left ( \textstyle\begin{array}{@{}c@{}} X^{\prime} \\ G(G^{*}G)^{-1}Y^{\prime}+Y^{\prime\prime} \\ G(G^{*}G)^{-1}Z^{\prime}+Z^{\prime\prime} \end{array}\displaystyle \right ). $$

For the second case, the matrix $GG^{*}$ is of full rank. We set

$$ \left ( \textstyle\begin{array}{@{}c@{}} X^{\prime} \\ Y^{\prime} \\ Z^{\prime} \\ X^{\prime\prime} \end{array}\displaystyle \right ) = \left ( \textstyle\begin{array}{@{}c@{}} GX \\ Y \\ Z \\ (I_{n}-G^{*}(GG^{*})^{-1}G)X \end{array}\displaystyle \right ), $$

where $X^{\prime\prime}$ is the unique solution of the following linear SDE:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{\prime\prime} =(I_{n}-G^{*}(GG^{*})^{-1}G)\phi _{t}\,dt+(I^{n}-G^{*}(GG^{*})^{-1}G)\varphi_{t}\,dM_{t}, \\ X_{0}^{\prime\prime} =(I_{n}-G^{*}(GG^{*})^{-1}G)x. \end{array}\displaystyle \right . $$

The triple $(X^{\prime},Y^{\prime},Z^{\prime})$ solves the FBSDE

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{\prime} =[-c_{2}^{\prime}GG^{*}Y_{t}^{\prime}+G\phi _{t}]\,dt+[-c_{2}^{\prime}GG^{*}Z_{t}^{\prime}+G\psi_{t}]\,dM_{t}, \\ -dY_{t}^{\prime} =[c_{2}X_{t}^{\prime}+\gamma_{t}]\,dt-Z_{t}^{\prime}\,dM_{t}, \\ X_{0}^{\prime}=Gx,\qquad Y_{T}^{\prime}=\lambda X_{T}^{\prime}+\xi. \end{array}\displaystyle \right . $$

(7)

In order to solve this equation, we introduce the following $m\times m$-symmetric matrix-valued matrix-Raccati equation:

$$ \left \{ \textstyle\begin{array}{@{}l} {-}\dot{K}(t) =c_{2}I_{m}-c_{2}^{\prime}KGG^{*}K,\quad t\in[0,T],\\ K(T) =\lambda I_{m}. \end{array}\displaystyle \right . $$

It is well known that this equation has a unique nonnegative solution $K \in C^{1}([0,T);S^{m})$, where $S^{m}$ stands for the space of all $m\times m$-symmetric matrices. We then look for the solution $(p,q) \in M^{2}(0,T;R^{m+m\times d})$ of the following linear simple BSDE:

$$ \left \{ \textstyle\begin{array}{@{}l} -dp_{t} =[-c_{2}^{\prime}K(t)GG^{*}p_{t}+K(t)G\phi_{t}+\gamma _{t}]\,dt+(K(t)G\psi_{t}-(I_{m}+c_{2}^{\prime}K(t)GG^{*})q_{t})\,dM_{t},\\ \quad t\in[0,T], \\ p_{T} =\xi. \end{array}\displaystyle \right . $$

We now let $X_{t}^{\prime}$ be the solution of the following SDE:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{\prime}=[-c_{2}^{\prime}GG^{*}(K(t)X_{t}^{\prime}+\gamma_{t})+G\phi _{t}]\,dt+[G\psi_{t}-c_{2}^{\prime}GG^{*}q_{t}]\,dM_{t}, \\ X_{0}^{\prime} =Gx. \end{array}\displaystyle \right . $$

Then we can easily check that $(X_{t}^{\prime},Y_{t}^{\prime},Z_{t}^{\prime})=(X_{t}^{\prime},K(t)X_{t}^{\prime}+p_{t},q_{t})$ is a solution of FBSDE (7). Once $(X^{\prime},X^{\prime\prime}, Y^{\prime},Z^{\prime})$ are resolved, then by the definition we can obtain the triple $(X,Y,Z)$ uniquely as

$$ \left ( \textstyle\begin{array}{@{}c@{}} X \\ Y \\ Z \end{array}\displaystyle \right ) = \left ( \textstyle\begin{array}{@{}c@{}} G^{*}(GG^{*})^{-1}X^{\prime}+X^{\prime\prime}\\ Y^{\prime} \\ Z^{\prime} \end{array}\displaystyle \right ). $$

The proof is complete. □

Lemma 3

We assume Assumptions 1 and 2. If for an $l_{0}\in[0,1)$, there exists a solution $(X^{l_{0}},Y^{l_{0}},Z^{l_{0}})$ of FBSDE (3), then there exists a positive constant $\delta_{0}$ such that for each $\delta\in [0,\delta_{0}]$, there exists a solution $(X^{l_{0}+\delta },Y^{l_{0}+\delta},Z^{l_{0}+\delta})$ of FBSDE (3) for $l=l_{0}+\delta$.

Proof

Since for all $\phi\in M^{2}(0,T; \mathbb{R}^{n})$, $\gamma\in M^{2}(0,T; \mathbb{R}^{m})$, $\psi\in M^{2}(0,T; \mathbb{R}^{n\times d})$, $\xi\in L^{2}(\Omega, \mathscr{F}_{T}, \mathbb{P})$, $x\in\mathbb{R}^{n}$, $l_{0}\in[0,1)$, there exists a unique solution of (3), for every $U_{s}=(X_{s}, Y_{s}, Z_{s})\in M^{2}(0,T; \mathbb{R}^{n+m+m\times d})$, there exists a unique triple $u_{s}=(x_{s}, y_{s}, z_{s})\in M^{2}(0,T; \mathbb{R}^{n+m+m\times d})$ satisfying the following FBSDE:

$$ \left \{ \textstyle\begin{array}{@{}l} dx_{t} =[(1-l_{0})c_{2}^{\prime}(-G^{*}y_{t})+l_{0}b(t, u_{t})+\delta b(t, U_{t})+\delta c_{2}^{\prime}(G^{*}Y_{t})+\phi_{t}]\,dt \\ \hphantom{dx_{t} =}{} +[(1-l_{0})c_{2}^{\prime}(-G^{*}z_{t})+l_{0}\sigma(t, u_{t})+\delta \sigma(t, U_{t})+\delta c_{2}^{\prime}(G^{*}Z_{t})+\psi_{t}]\,dM_{t}, \\ -dy_{t} =[(1-l_{0})c_{2}Gx_{t}+l_{0}f(t, u_{t})+\delta (-c_{2}GX_{t})+\delta f(t, U_{t})+\gamma_{t}]\,dt-z_{t}\,dM_{t}, \\ x_{0} =x, \qquad y_{T}=l_{0}\Phi(x_{T})+(1-l_{0})Gx_{T}+\delta(\Phi (X_{T})-GX_{T})+\xi. \end{array}\displaystyle \right . $$

We want to prove that the mapping defined by

$$\begin{aligned} I_{l_{0}+\delta}(U\times X_{T})&=u\times x_{T}: M^{2}\bigl(0,T; \mathbb {R}^{n+m+m\times d}\bigr)\times L^{2}(\Omega,\mathscr{F}_{T}, \mathbb{P}) \\ &\rightarrow M^{2}\bigl(0,T; \mathbb{R}^{n+m+m\times d}\bigr)\times L^{2}(\Omega,\mathscr{F}_{T}, \mathbb{P}) \end{aligned}$$

is a contraction.

Let $U^{\prime}\times X_{T}^{\prime}=(X^{\prime},Y^{\prime},Z^{\prime})\times X_{T}^{\prime} \in M^{2}(0,T; \mathbb{R}^{n+m+m\times d})\times L^{2}(\Omega,\mathscr{F}_{T}, \mathbb{P})$ and $u^{\prime}\times x_{T}^{\prime}=I_{l_{0}+\delta}(U^{\prime} \times X_{T}^{\prime})$. We set

$$\begin{aligned} \hat{U} =&(\hat{X},\hat{Y},\hat{Z})=\bigl(X-X^{\prime},Y-Y^{\prime},Z-Z^{\prime} \bigr), \\ \hat{u} =&(\hat{x},\hat{y},\hat{z})=\bigl(x-x^{\prime},y-y^{\prime},z-z^{\prime} \bigr), \\ \hat{f}_{s} =&f(s,U_{s})-f\bigl(s,U_{s}^{\prime} \bigr), \qquad\hat {b}_{s}=b(s,U_{s})-b\bigl(s,U_{s}^{\prime} \bigr),\qquad\hat{\sigma}_{s}=\sigma (s,U_{s})-\sigma \bigl(s,U_{s}^{\prime}\bigr), \\ \bar{f}_{s} =&f(s,u_{s})-f\bigl(s,u_{s}^{\prime} \bigr), \qquad\bar {b}_{s}=b(s,u_{s})-b\bigl(s,u_{s}^{\prime} \bigr), \qquad\bar{\sigma}_{s}=\sigma (s,u_{s})-\sigma \bigl(s,u_{s}^{\prime}\bigr). \end{aligned}$$

Then

$$ \left \{ \textstyle\begin{array}{@{}l} d\hat{x}_{s} = [(1-l_{0})c_{2}^{\prime}(-G^{*}\hat{y_{s}})+l_{0}\bar {b}_{s}+\delta\hat{b}_{s}+\delta c_{2}^{\prime}G^{*}\hat{Y_{s}}]\,ds \\ \hphantom{d\hat{x}_{s} =}{} + [(1-l_{0})c_{2}^{\prime}(-G^{*}\hat{z_{s}})+l_{0}\bar{\sigma }_{s}+\delta\hat{\sigma}_{s}+\delta c_{2}^{\prime}G^{*}\hat{Z_{s}}]\,dM_{s}, \\ -d\hat{y}_{s} = [(1-l_{0})c_{2}G\hat{x}_{s}+l_{0}\bar{f}_{s}+\delta (-c_{2}G\hat{X}_{s})+\delta\hat{f}_{s}]\,ds-\hat{z}_{s}\,dM_{s}, \\ \hat{x}_{0}=0, \qquad\hat{y}_{T} = l_{0}(\Phi(x_{T})-\Phi (x_{T}^{\prime}))+(1-l_{0})G\hat{x}_{T}+\delta(\Phi(x_{T})-\Phi (x_{T}^{\prime})-G\hat{x}_{T}). \end{array}\displaystyle \right . $$

Using the Itô product rule of semimartingales for the product $(G\hat{x}_{s}, \hat{y}_{s})$, taking expectations, and evaluating at $t=T$ yield

$$\begin{aligned} &l_{0}E\bigl(\Phi(x_{T})-\Phi\bigl(x_{T}^{\prime} \bigr), G\hat {x}_{T}\bigr)+(1-l_{0})E(G\hat{x}_{T}, G\hat{x}_{T})+\delta E\bigl(\Phi (X_{T})-\Phi \bigl(X_{T}^{\prime}\bigr)-G\hat{X}_{T}, G \hat{x}_{T}\bigr) \\ &\quad =l_{0}E \int_{0}^{T}\bigl[F(s,u_{s})-F \bigl(s,u_{s}^{\prime}\bigr),\hat {u}_{s} \bigr]\,ds+l_{0}E \int_{0}^{T}\bigl[H(s,u_{s})-H \bigl(s,u_{s}^{\prime}\bigr),\hat{u}_{s}\bigr]\,d\langle M,M \rangle_{s} \\ &\qquad{} -(1-l_{0})E \int_{0}^{T}\bigl[c_{2}|G\hat {x}_{s}|^{2}+c_{2}^{\prime}\bigl|G^{*} \hat{y}_{s}\bigr|^{2}\bigr]\,ds-(1-l_{0})E \int _{0}^{T}\bigl[c_{2}^{\prime}\bigl|G^{*} \hat{z}_{s}\bigr|^{2}\bigr]\,d\langle M,M \rangle_{s} \\ &\qquad{} +\delta E \int_{0}^{T}\bigl[c_{2}(G \hat{x}_{s},G\hat {X}_{s})+c_{2}^{\prime} \bigl(G^{*}\hat{y}_{s},G^{*}\hat{Y}_{s} \bigr)+\bigl(\hat {x}_{s},-G^{*}\hat{f}_{s} \bigr)+\bigl(G^{*}\hat{y}_{s}, \hat{b}_{s}\bigr) \bigr]\,ds \\ &\qquad{} +\delta E \int_{0}^{T}\bigl[c_{2}^{\prime} \bigl(G^{*}\hat{Z}_{s},G^{*}\hat {z}_{s}\bigr)+(\hat{z}_{s},G\hat{\sigma}_{s}) \bigr]\,d\langle M,M \rangle_{s}. \end{aligned}$$

By Assumptions 1 and 2, since $c_{2}>0$ and $c_{3}>0$, we obtain

$$\begin{aligned} & \bigl(l_{0}c_{3}+(1-l_{0})\bigr)E|G \hat{x}_{T}|^{2}+c_{2}E \int _{0}^{T}|G\hat{x}_{s}|^{2}\,ds\\ &\qquad{}+c_{2}^{\prime}E \int_{0}^{T}\bigl|G^{*}\hat {y}_{s}\bigr|^{2}\,ds+l_{0}c_{2}^{\prime}E \int_{0}^{T}\bigl|G^{*}\hat{z}_{s}\bigr|^{2}\,ds \\ &\qquad{} +l_{0}c_{2}^{\prime}E \int_{0}^{T}\bigl|G^{*}\hat{y}_{s}\bigr|^{2}\,d\langle M,M \rangle_{s}+l_{0}c_{2}E \int_{0}^{T}|G\hat{x}_{s}|^{2}\,d\langle M, M \rangle_{s}\\ &\qquad{}+c_{2}^{\prime}E \int_{0}^{T}\bigl|G^{*}\hat{z}_{s}\bigr|^{2}\,d\langle M, M \rangle_{s} \\ &\quad \leq\delta c_{4}E \int_{0}^{T}\bigl(|\hat{U}_{s}|^{2}+| \hat {u}_{s}|^{2}\bigr)\,ds+\delta c_{4}E \int_{0}^{T}\bigl(|\hat{U}_{s}|^{2}+| \hat {u}_{s}|^{2}\bigr)\,d\langle M,M \rangle_{s} \\ &\qquad{} +\delta c_{4}E|\hat{X}_{T}|^{2}+\delta c_{4}E|\hat{x}_{T}|^{2}. \end{aligned}$$

For the difference of the solutions $(\hat{y},\hat {z})=(y-y^{\prime},z-z^{\prime})$, we apply the usual technique to the BSDE and derive that

$$\begin{aligned} &E \int_{0}^{T}|\hat{y}_{s}|^{2}\,ds+E \int_{0}^{T}|\hat {z}_{s}|^{2}\,d\langle M,M \rangle_{s} \\ &\quad \leq c_{4}\delta E \int_{0}^{T}|\hat{U}_{s}|^{2}\,ds+c_{4} \delta E|\hat{X}_{T}|^{2}+c_{4}E \int_{0}^{T}|\hat{x}_{s}|^{2}\,ds+c_{4}E| \hat {x}_{T}|^{2}, \\ &E \int_{0}^{T}|\hat{y}_{s}|^{2}\,ds+E \int_{0}^{T}|\hat{z}_{s}|^{2}\,d\langle M,M \rangle_{s} \\ &\quad\leq c_{4}\delta E \int_{0}^{T}|\hat{U}_{s}|^{2}\,ds+c_{4} \delta E|\hat {X}_{T}|^{2}+c_{4}E \int_{0}^{T}|\hat{x}_{s}|^{2}\,ds+c_{4}E| \hat {x}_{T}|^{2}. \end{aligned}$$

Similarly, for the difference of the solution $\hat{x}=x-x^{\prime}$, we apply the usual technique to the forward part:

$$\begin{aligned}& \begin{aligned}[b] \sup_{0\leq s \leq T}|\hat{x}_{s}|^{2}\leq{}& c_{4}E \int_{0}^{T}\bigl(|\hat{y}_{s}|^{2}+| \hat{z}_{s}|^{2}\bigr)\,ds+c_{4} E \int _{0}^{T}|\hat{u}_{s}|^{2}\,d\langle M,M \rangle_{s} \\ &{} +c_{4}\delta E \int_{0}^{T}|\hat {U}_{s}|^{2}\,ds+c_{4} \delta E \int_{0}^{T}|\hat{U}_{s}|^{2}\,d\langle M,M \rangle_{s}, \end{aligned}\\& \begin{aligned}[b] E \int_{0}^{T}|\hat{x}_{s}|^{2}\,ds \leq{}& c_{4}TE \int_{0}^{T}\bigl(|\hat {y}_{s}|^{2}+| \hat{z}_{s}|^{2}\bigr)\,ds+c_{4} TE \int_{0}^{T}|\hat {u}_{s}|^{2}\,d\langle M,M \rangle_{s} \\ &{} +c_{4}\delta TE \int_{0}^{T}|\hat {U}_{s}|^{2}\,ds+c_{4} \delta T E \int_{0}^{T}|\hat{U}_{s}|^{2}\,d\langle M,M \rangle_{s}. \end{aligned} \end{aligned}$$

Here the constant $c_{4}$ depends on the Lipschitz constants, and so do G, $c_{2}$, $c_{2}^{\prime}$, and T.

Combing the above estimates, we get

$$\begin{aligned} & E \int_{0}^{T}|\hat{u}_{s}|^{2}\,ds+E \int_{0}^{T}|\hat {u}_{s}|^{2}\,d\langle M,M \rangle_{s}+E|\hat{x}_{T}|^{2} \\ &\quad \leq c_{5} \delta\biggl\{ E \int_{0}^{T}|\hat{U}_{s}|^{2}\,ds+E \int _{0}^{T}|\hat{U}_{s}|^{2}\,d\langle M,M \rangle_{s}+E|\hat{X}_{T}|^{2}\biggr\} , \end{aligned}$$

where the constant $c_{5}$ depends on $c_{4}$, G, $c_{2}$, and $c_{3}$. If we choose $\delta_{0}=\frac{1}{2c_{5}}$, then clearly for each fixed $\delta\in[0, \delta_{0}]$, the mapping $I_{\alpha+\delta}$ is a contraction in the sense that

$$\begin{aligned} & E \int_{0}^{T}|\hat{u}_{s}|^{2}\,ds+E \int_{0}^{T}|\hat {u}_{s}|^{2}\,d\langle M,M \rangle_{s}+E|\hat{x}_{T}|^{2} \\ &\quad \leq\frac{1}{2} \biggl\{ E \int_{0}^{T}|\hat{U}_{s}|^{2}\,ds+E \int _{0}^{T}|\hat{U}_{s}|^{2}\,d\langle M,M \rangle_{s}+E|\hat{X}_{T}|^{2}\biggr\} . \end{aligned}$$

This indicates that this mapping has a unique fixed point $(u^{l_{0}+\delta})=(x^{l_{0}+\delta}, y^{l_{0}+\delta}, z^{l_{0}+\delta})$, which is a solution of FBSDE (3) for $l=l_{0}+\delta$. The proof is complete. □

Proof of existence

From Lemma 2 we immediately see that, when $\lambda=1$ in FBSDE (5), FBSDEs (3) for $l=0$ (that is, FBSDE (4)) has a unique solution. It then follows from Lemma 3 that there exists a positive constant $\delta_{0}$ depending on Lipschitz constants, $c_{2}$, $c_{2}^{\prime}$, $c_{3}$, and T such that, for each $\delta\in[0,\delta_{0}]$, FBSDEs (3) for $l=l_{0}+\delta$ has a unique solution. We can repeat this process for N times with $1\leq N \delta_{0}<1+\delta_{0}$. It then follows that, in particular, FBSDE (3) for $l=1$ with $\xi \equiv0$ has a unique solution. The proof is complete. □

5 Another existence and uniqueness theorem

Next, we give another existence and uniqueness theorem. First, we give another assumption.

Assumption 3

There exist constants $c_{2}$, $c_{2}^{\prime}$, $c_{3}$ such that

$$\begin{aligned}& \bigl[F\bigl(t,u^{1}\bigr)-F\bigl(t,u^{2}\bigr), u^{1}-u^{2}\bigr] \geq c_{2}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2}+c_{2}^{\prime} \bigl(\bigl|G^{*}\bigl(y^{1}-y^{2} \bigr)\bigr|^{2}+\bigl|G^{*}\bigl(z^{1}-z^{2} \bigr)\bigr|^{2}\bigr), \\& \bigl[H\bigl(t,u^{1}\bigr)-H\bigl(t,u^{2}\bigr), u^{1}-u^{2}\bigr] \geq c_{2}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2}+c_{2}^{\prime} \bigl(\bigl|G^{*}\bigl(y^{1}-y^{2} \bigr)\bigr|^{2}+\bigl|G^{*}\bigl(z^{1}-z^{2} \bigr)\bigr|^{2}\bigr), \\& \quad\forall u^{1}=\bigl(x^{1},y^{1},z^{1} \bigr), u^{2}=\bigl(x^{2},y^{2},z^{2} \bigr) \in\mathbb{R}^{n}\times\mathbb{R}^{m}\times \mathbb{R}^{m\times d},\ \mathbb{P}\mbox{-a.s., a.e. }t\in\mathbb{R}^{+}, \end{aligned}$$

and

$$\bigl(\Phi\bigl(x^{1}\bigr)-\Phi\bigl(x^{2}\bigr), G \bigl(x^{1}-x^{2}\bigr)\bigr)\leq -c_{3}\bigl|G \bigl(x^{1}-x^{2}\bigr)\bigr|^{2},\quad \forall x^{1}\in\mathbb{R}^{n}, \forall x^{2}\in \mathbb{R}^{n}, $$

where $c_{2}$, $c_{2}^{\prime}$, and $c_{3}$ are given positive constants.

Theorem 3

Let Assumptions 1 and 3 hold. Then there exists a unique adapted solution $(X,Y,Z)$ for FBSDE (1).

Proof of uniqueness

Using the same procedure as in the proof of uniqueness of Theorem 2, by Assumption 3 we get

$$\begin{aligned} & E(\hat{Y_{T}},G\hat{X_{T}}) \\ &\quad=E\bigl(\Phi\bigl(X_{T}^{1}\bigr)-\Phi \bigl(X_{T}^{2}\bigr),G\bigl(X_{T}^{1}-X_{T}^{2} \bigr)\bigr) \\ &\quad =E \int_{0}^{T}\bigl[F\bigl(s,U^{1}\bigr)-F \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,ds \\ &\qquad{} +E \int_{0}^{T}\bigl[H\bigl(s,U^{1}\bigr)-H \bigl(s,U^{2}\bigr),U^{1}-U^{2}\bigr]\,d\langle M,M \rangle_{s} \\ &\quad \geq c_{2}E \int_{0}^{T}|G\hat{X}|^{2}\,ds+c_{2}^{\prime}E \int _{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,ds \\ &\qquad{} +c_{2}E \int_{0}^{T}|G\hat{X}|^{2}\,d\langle M,M \rangle _{s}+c_{2}^{\prime}E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,d\langle M,M \rangle_{s}. \end{aligned}$$

Thus, we deduce that

$$\begin{aligned} & c_{2}\biggl(E \int_{0}^{T}|G\hat{X}|^{2}\,ds+E \int _{0}^{T}|G\hat{X}|^{2}\,d\langle M,M \rangle_{s}\biggr) \\ &\qquad{} +c_{2}^{\prime}\biggl[E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat {Z}\bigr|^{2}\bigr)\,ds+E \int_{0}^{T}\bigl(\bigl|G^{*} \hat{Y}\bigr|^{2}+\bigl|G^{*}\hat{Z}\bigr|^{2}\bigr)\,d\langle M,M \rangle_{s}\biggr]+c_{3}|G\hat{X}|^{2} \\ &\quad \leq0. \end{aligned}$$

Since $c_{2}$, $c_{2}^{\prime}$, and $c_{3}$ are given positive constants, we have $|G\hat{X}|^{2}\equiv0$, $|G^{*}\hat{Y}|^{2}\equiv0$, $|G^{*}\hat {Z}|^{2}\equiv0$. So $\hat{X}_{s}^{1}\equiv\hat{X}_{s}^{2}$, $\hat {Y}_{s}^{1}\equiv\hat{Y}_{s}^{2}$, $\hat{Z}_{s}^{1}\equiv\hat{Z}_{s}^{2}$, $d \langle M,M \rangle_{t}\times P$-a.s. □

We now consider the following family of FBSDEs parameterized by $l\in[0,1]$:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{l} =[(1-l)c_{2}^{\prime}(G^{*}Y_{t}^{l})+lb(t, U_{t}^{l})+\phi _{t}]\,dt\\ \hphantom{dX_{t}^{l} =}{}+[l\sigma(t, U_{t}^{l})+(1-l)c_{2}^{\prime}(G^{*}Z_{t}^{l})+\psi _{t}]\,dM_{t}, \\ -dY_{t}^{l} =[-(1-l)c_{2}GX_{t}^{l}+lf(t, U_{t}^{l})+\gamma _{t}]\,dt-Z_{t}^{l}\,dM_{t}, \\ X_{0}^{l} =x,\qquad Y_{T}^{l}=l\Phi(X_{T}^{l})+(1-l)GX_{T}^{l}+\xi, \end{array}\displaystyle \right . $$

(8)

where ϕ, ψ, and γ are given processes in $M^{2}(0,T)$ with values in $R^{n}$, $R^{n\times d}$, and $R^{m}$, respectively. Clearly, when $l=1$ and $\xi\equiv0$, the existence of a solution of FBSDE (8) implies that of FBSDE (1). When $l=0$ in FBSDEs (8), it is

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t}^{0} =[c_{2}^{\prime}G^{*}Y_{t}^{0}+\phi _{t}]\,dt+[c_{2}^{\prime}G^{*}Z_{t}^{0}+\psi_{t}]\,dM_{t}, \\ -dY_{t}^{0} =[-c_{2}GX_{t}^{0}+\gamma_{t}]\,dt-Z_{t}^{0}\,dM_{t}, \\ X_{0}^{l}=x,\qquad Y_{T}^{0}=GX_{T}^{0}+\xi. \end{array}\displaystyle \right . $$

(9)

For proving the existence part of the theorem, we first need the following two lemmas.

Lemma 4

The following equation has a unique solution:

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t} =[c_{2}^{\prime}G^{*}Y_{t}+\phi_{t}]\,dt+[c_{2}^{\prime}G^{*}Z_{t}+\psi _{t}]\,dM_{t}, \\ -dY_{t} =[-c_{2}GX_{t}+\gamma_{t}]\,dt-Z_{t}\,dM_{t} ,\\ X_{0} =x,\qquad Y_{T}=\lambda GX_{T}+\xi, \end{array}\displaystyle \right . $$

(10)

where λ is a nonnegative constant.

Lemma 5

Let Assumptions 1 and 3 hold. If for an $l_{0}\in[0,1)$, there exists a solution $(X^{l_{0}},Y^{l_{0}},Z^{l_{0}})$ of FBSDE (8), then there exists a positive constant $\delta_{0}$ such that for each $\delta\in [0,\delta_{0}]$, there exists a solution $(X^{l_{0}+\delta },Y^{l_{0}+\delta},Z^{l_{0}+\delta})$ of FBSDE (8) for $l=l_{0}+\delta$.

Remark 1

The proofs of Lemma 4 and Lemma 5 are similar to those of Lemma 2 and Lemma 3.

Proof of existence

From Lemma 4 we immediately see that when $\lambda=1$ in FBSDE (10), FBSDE (8) for $l=0$ (that is, FBSDE (9)) has a unique solution. It then follows from Lemma 5 that there exists a positive constant $\delta_{0}$ depending on the Lipschitz constants, $c_{2}$, $c_{2}^{\prime}$, $c_{3}$, and T such that, for each $\delta\in[0,\delta_{0}]$, FBSDE (8) for $l=l_{0}+\delta$ has a unique solution. We can repeat this process for N times with $1\leq N \delta_{0}<1+\delta_{0}$. It then follows that, in particular, FBSDEs (8) for $l=1$ with $\xi\equiv0$ has a unique solution.

The proof is complete. □

6 Application to stochastic optimal control problems

In this section, we give an application of the theorem to the stochastic maximum principle in optimal control problems through an example of the linear quadratic case. Consider a continuous-time finite-state Markov chain $m=\{m_{t},t\in [0,T]\}$. We identify the states of the process with the unit vectors $e_{i}$ in $\mathbb{R}^{k}$, where k is the number of states of the chain. The rate matrix of $m_{t}$ is denoted by $\mathcal{A}_{t}$.

All the stochastic processes are defined on the filtered probability space $(\Omega,\mathscr{F},\mathscr{F}_{t},\mathbb{P})$, where $\{\mathscr{F}_{t}\}$ is the completed natural filtration generated by the σ-fields $\sigma(\{m_{s},s\leq t \},F\in\mathscr{F}_{T}:\mathbb{P}(F)=0)$, and $\mathscr{F}=\mathscr{F}_{T}$. As noted before, the martingale $M_{t}=m_{t}-m_{0}-\int_{0}^{t}\mathcal {A}_{s}m_{s}\,ds$ is used to drive the controlled state process $X_{t}$. Next, we define the function $\Lambda:[0,T]\times\{e_{i}|i=1,2,\ldots,k\} \rightarrow \mathcal{S}^{k}$ by

$$\Lambda(s,e_{i}):=\operatorname{diag}(\mathcal {A}_{s}e_{i})- \operatorname{diag}(e_{i})\mathcal{A}_{s}^{*}- \mathcal {A}_{s}\operatorname{diag}(e_{i}). $$

Note that

$$\begin{aligned} \langle M,M\rangle_{t}&= \int_{0}^{t}\bigl[\operatorname{diag}(\mathcal {A}_{s}m_{s})-\operatorname{diag}(m_{s}) \mathcal{A}_{s}^{*}-\mathcal {A}_{s} \operatorname{diag}(m_{s})\bigr]\,ds \\ &= \int_{0}^{t}\Lambda(s,m_{s})\,ds. \end{aligned}$$

Consider the one-dimensional linear quadratic optimal control problem

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t} = [AX_{t}+Bu_{t}]\,dt+[CX_{t-}+Du_{t}]\,dM_{t}, \\ X_{0} =x, \end{array}\displaystyle \right . $$

(11)

and the cost function

$$ J\bigl(u(\cdot)\bigr)= E\biggl[\frac{1}{2} \int_{0}^{T}\bigl(QX_{t}^{2}+Ru_{t}^{2} \bigr)\,dt+\frac {1}{2}Gx_{T}^{2}\biggr], $$

(12)

where the parameters A, B, Q, R, G are positive constant numbers, and C, D are $\mathbb{R}^{1\times k}$ vectors.

Denote $\mathcal {U}=\{u:[0,T]\times\Omega\rightarrow\mathbb{R}\mid u \mbox{ is } \{ \mathscr{F}_{t}\}\mbox{-predictable}, E\int_{0}^{T}|u(s)|^{2}\,ds<+\infty \}$. The stochastic control problem is to find an optimal control $u^{*}\in\mathcal{U}$ such that

$$J\bigl(u^{*}\bigr)=\inf_{u\in\mathcal {U}}J(u). $$

In order to solve the problem, we define the Hamiltonian $\mathcal {H}:[0,T]\times\mathbb{R} \times\mathbb{R}\times\mathbb{R}\times \mathbb{R}^{1\times k}\times\{e_{i}|i=1,2,\ldots,k\} \rightarrow \mathbb{R}$ by

$$\mathcal{H}(t,x,u,p,s,e_{i})=\frac{1}{2}Qx^{2}+ \frac {1}{2}Ru^{2}+(Ax+Bu)p+(Cx+Du)\Lambda(t,e_{i}) s^{*}. $$

The adjoint equation is given by the following BSDE:

$$ \left \{ \textstyle\begin{array}{@{}l} dp_{t} = -\mathcal{H}_{x}(t,X_{t},u_{t},p_{t},s_{t},m_{t})\,dt+s_{t}\,dM_{t} \\ \hphantom{dp_{t}}=-(QX_{t}+Ap_{t}+C_{t} \Lambda(t,m_{t}) s_{t}^{*})\,dt+s_{t}\,dM_{t}, \\ p_{T} =GX_{T}. \end{array}\displaystyle \right . $$

(13)

Then the sufficient stochastic maximum principle is stated as follows.

Theorem 4

Given the assumptions on the parameters above, the coupled FBSDE

$$ \left \{ \textstyle\begin{array}{@{}l} dX_{t} = [AX_{t}-BR^{-1}Bp_{t}-BR^{-1}D\Lambda(t,m_{t}) s_{t}^{*}]\,dt,\\ \hphantom{dX_{t} =}{}+[CX_{t-}-DR^{-1}Bp_{t-}-DR^{-1}D\Lambda(t,m_{t}) s_{t}^{*}]\,dM_{t}, \\ dp_{t}=-(QX_{t}+Ap_{t}+C_{t} \Lambda(t,m_{t}) s_{t}^{*})\,dt+s_{t}\,dM_{t},\\ X_{0} =x,\qquad p_{T} =GX_{T}, \end{array}\displaystyle \right . $$

(14)

exists a unique solution $(\hat{X}_{t},\hat{p}_{t},\hat{s}_{t})$. Moreover, the stochastic linear quadratic problem is solvable with the optimal controlled state process $\hat{X}_{t}$ and the optimal control defined by

$$ \hat{u}_{t}=-R^{-1}\bigl(B \hat{p}_{t-}+D \Lambda(t,m_{t}) \hat{s}_{t}^{*} \bigr). $$

(15)

Proof

First, it is easy to check that FBSDE (14) satisfies Assumptions 1 and 3. So, according to Theorem 3, there exists a unique adapted solution $(\hat{X}_{t},\hat{p}_{t},\hat{s}_{t})\in\mathcal {M}^{2}(0,T;\mathbb{R}\times\mathbb{R}\times\mathbb{R}^{1\times k})$. So, according to (15), $\hat{u}\in\mathcal {U}$, and X̂ is the corresponding state process. Next, for any fixed $u\in\mathcal{U}$ with the corresponding state process $X_{t}$, consider

$$J(u)-J(\hat{u})=E\biggl[\frac{1}{2} \int_{0}^{T}\bigl(QX_{t}^{2}+Ru_{t}^{2}-Q \hat {X}_{t}^{2}-R\hat{u}_{t}^{2}\bigr)\,dt+ \frac{1}{2}\bigl(GX_{T}^{2}-G\hat{X}_{T}^{2} \bigr)\biggr]. $$

Since $G>0$, we have

$$\begin{aligned} &E\biggl[\frac{1}{2}\bigl(GX_{T}^{2}-G \hat{X}_{T}^{2}\bigr)\biggr] \\ &\quad\geq E\bigl[(X_{T}-\hat{X_{T}})G\hat{X_{T}} \bigr] \\ &\quad= E\bigl[(X_{T}-\hat{X_{T}})\hat{p_{T}} \bigr] \\ &\quad=E\biggl[ \int_{0}^{T}(X_{t-}-\hat{X_{t-}})\,d\hat{p_{t}}+ \int_{0}^{T}\hat {p_{t-}}d(X_{t}- \hat{X_{t}})+\sum_{0< t\leq T} \Delta(X_{t}-\hat {X_{t}})\Delta p_{t}\biggr] \\ &\quad=E\biggl[ \int_{0}^{T}(X_{t-}-\hat{X_{t-}}) \bigl(-\mathcal{H}_{x}(t,\hat{X}_{t-},\hat {u}_{t},\hat{p}_{t-},\hat{s}_{t},m_{t}) \bigr)\,dt+ \int_{0}^{T}(X_{t-}-\hat{X_{t-}}) \hat {s}\,dM_{t} \\ &\qquad{}+ \int_{0}^{T}\hat{p}_{t-} \bigl[A(X_{t-}-\hat{X_{t-}})+B(u_{t}-\hat {u_{t}})\bigr]\,dt+ \int_{0}^{T}\hat{p}_{t-} \bigl[C(X_{t-}-\hat{X_{t-}})+D(u_{t}-\hat {u_{t}})\bigr]\,dM_{t} \\ &\qquad{} +\sum_{0< t\leq T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat {u_{t}})\bigr]\Delta M_{t}s_{t} \Delta M_{t}\biggr]. \end{aligned}$$

Notice that

$$\begin{aligned} &\sum_{0< t\leq T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat {u_{t}})\bigr]\Delta M_{t}s_{t} \Delta M_{t} \\ &\quad= \int_{0}^{T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat{u_{t}}) \bigr]\,d[M,M]_{t} s_{t}^{*} \\ &\quad=L_{t}+ \int_{0}^{T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat{u_{t}}) \bigr]\,d\langle M,M\rangle_{t} s_{t}^{*} \\ &\quad=L_{t}+ \int_{0}^{T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat{u_{t}})\bigr]\Lambda (t,m_{t}) s_{t}^{*}\,dt \end{aligned}$$

for some local martingale L (see [5]). So

$$\begin{aligned} &E\biggl[\frac{1}{2}\bigl(GX_{T}^{2}-G \hat{X}_{T}^{2}\bigr)\biggr] \\ &\quad\geq E\biggl[ \int_{0}^{T}(X_{t-}-\hat{X_{t-}}) \bigl(-\mathcal{H}_{x}(t,\hat {X}_{t-},\hat{u}_{t}, \hat{p}_{t-},\hat{s}_{t},m_{t})\bigr)\,dt \\ &\qquad{}+ \int_{0}^{T}\hat{p}_{t-} \bigl[A(X_{t-}-\hat{X_{t-}})+B(u_{t}- \hat{u_{t}})\bigr]\,dt \\ &\qquad{}+ \int_{0}^{T}\bigl[C(X_{t-}- \hat{X_{t-}})+D(u_{t}-\hat{u_{t}})\bigr]\Lambda (t,m_{t}) s_{t}^{*}\,dt \biggr]. \end{aligned}$$

By the definition of $\mathcal{H}$ we have

$$\begin{aligned} &E\biggl[\frac{1}{2} \int_{0}^{T}\bigl(QX_{t}^{2}+Ru_{t}^{2}-Q \hat{X}_{t}^{2}-R\hat{u}_{t}^{2}\bigr)\,dt \biggr] \\ &\quad=E\biggl[\frac{1}{2} \int_{0}^{T}\bigl(QX_{t-}^{2}+Ru_{t}^{2}-Q \hat{X}_{t-}^{2}-R\hat {u}_{t}^{2} \bigr)\,dt\biggr] \\ &\quad=E\biggl[ \int_{0}^{T} \bigl\{ \mathcal{H}(t,X_{t-},u_{t}, \hat{p}_{t-},\hat {s}_{t},m_{t})-\mathcal{H}(t, \hat{X}_{t-},\hat{u}_{t},\hat{p}_{t-},\hat {s}_{t},m_{t}) \\ &\qquad{}-\bigl[A(X_{t-}-\hat{X}_{t-})+B(u_{t}- \hat{u}_{t})\bigr]\hat{p}_{t-} -\bigl[C(X_{t-}- \hat{X}_{t-})+D(u_{t}-\hat{u}_{t})\bigr] \Lambda(t,m_{t}) \hat{s}_{t}^{*} \bigr\} \,dt\biggr]. \end{aligned}$$

Adding the two equations, we have

$$\begin{aligned} J(u)-J(\hat{u}) \geq& E\biggl[ \int_{0}^{T}\bigl(\mathcal{H}(t,X_{t-},u_{t}, \hat {p}_{t-},\hat{s}_{t},m_{t})-\mathcal{H}(t, \hat{X}_{t-},\hat{u}_{t},\hat {p}_{t-}, \hat{s}_{t},m_{t}) \\ &{}-(X_{t-}-\hat{X}_{t-}) \mathcal{H}_{x}(t, \hat{X}_{t-},\hat{u}_{t},\hat {p}_{t}, \hat{s}_{t},m_{t})\bigr)\,dt\biggr] \\ =&E\biggl\{ \int_{0}^{T} \bigl[\mathcal{H}(t,X_{t-},u_{t}, \hat{p}_{t-},\hat {s}_{t},m_{t})-\mathcal{H}(t, \hat{X}_{t-},\hat{u}_{t},\hat{p}_{t-},\hat {s}_{t},m_{t}) \\ &{}-(X_{t-}-\hat{X}_{t-}) \bigl(Q\hat{X}_{t-}+A \hat{p}_{t-}+C\Lambda(t,m_{t}) \hat{s}_{t}\bigr) \bigr]\,dt\biggr\} \\ =&E\biggl[ \int_{0}^{T}\biggl(\frac{1}{2}QX_{t-}^{2}- \frac{1}{2}Q\hat {X}_{t-}^{2}-(X_{t-}- \hat{X}_{t-})Q\hat{X}_{t-} \\ &{}+\frac{1}{2}Ru_{t}^{2}+Bu_{t} \hat{p}_{t-}+Du_{t}\Lambda(t,m_{t}) \hat{s}_{t}^{*} \\ &{}-\frac{1}{2}R\hat{u}_{t}^{2}-B \hat{u}_{t}\hat{p}_{t-}-D\hat{u}_{t}\Lambda (t,m_{t})\hat{s}_{t}^{*}\biggr)\,dt\biggr] \\ \geq& 0. \end{aligned}$$

By the convexity of $\frac{1}{2}Qx^{2}$ and the fact that $\hat {u}_{t}=-R^{-1}(B \hat{p}_{t-}+D \Lambda(t,m_{t}) \hat{s}_{t}^{*})$ we get the minimum point of the function $F(u)=\frac{1}{2}Ru_{t}^{2}+Bu_{t}\hat{p}_{t-}+Du_{t}\Lambda(t,m_{t})\hat{s}_{t}^{*}$. Therefore, we conclude that

$$J(u)-J(\hat{u})\geq0, $$

which proves that û is optimal. □

Remark 2

Differently from the Brownian motion case, the optimal control and the Hamiltonian additionally depend on the state of the Markov chain. This is because Markov chains are discontinuous stochastic processes with finite variation and the quadratic processes of the martingales generated by Markov chains are quite different from the continuous-time diffusions.