- Research
- Open Access
- Published:
Solvability and optimal stabilization controls of discrete-time mean-field stochastic system with infinite horizon
Advances in Difference Equations volume 2020, Article number: 187 (2020)
Abstract
The paper addresses the optimal control and stabilization problems for the indefinite discrete-time mean-field system over infinite horizon. Firstly, we show the convergence of the generalized algebraic Riccati equations (GAREs) and establish their compact form GARE. By dealing with the GARE, we derive the existence of the maximal solution to the original GAREs along with the fact that the maximal solution is the stabilizing solution. Then, the maximal solution is employed to design the linear-quadratic (LQ) optimal controller and the optimal value of the control problem. Specifically, we deduce that under the assumption of exact observability, the mean-field system is \(L^{2}\)-stabilizable if and only if the GAREs have a solution, which is also the maximal solution. By semi-definite programming (SDP) method, the solvability of the GAREs is discussed. Our results generalize and improve previous results. Finally, some numerical examples are exploited to illustrate the validity of the obtained results.
1 Introduction
We are curious about the indefinite discrete-time mean-field LQ (MF-LQ) optimal control problems over infinite horizon. The system equation is the following stochastic difference equation (SDE):
where \(x_{k}\in \mathbb{R}^{n}\) and \(u_{k}\in \mathbb{R}^{m}\) are the state, control processes, respectively. \(\mathbb{E}\) is the expectation operator and \(\{w_{k}\}_{k\geq 0}\) is a martingale difference sequence, defined on a complete filtered probability space \((\varOmega ,\mathfrak{F},\{\mathfrak{F}_{k}\}_{k\geq 0},P)\), in the sense that \(\mathbb{E}[w_{k}|\mathfrak{F}_{k}]=0\), \(\mathbb{E}[w_{k}^{2}|\mathfrak{F}_{k}]=\sigma ^{2}\), where \(\mathfrak{F}_{k}\) is the σ-field generated by \(\{\xi ,w_{0},\ldots ,w_{k-1}\}\), \(\mathfrak{F}_{0}=\{\emptyset ,\varOmega \}\). The initial values ξ and \(w_{k}\) are assumed to be independent of each other. \(A, \bar{A}, C, \bar{C} \in \mathbb{R}^{n\times n}\) and \(B, \bar{B}, D, \bar{D} \in \mathbb{R}^{n\times m}\) are given deterministic matrices. Indeed, system (1.1) is also regarded as a mean-field SDE (MF-SDE). Mean-field theory has been developed to investigate the collective behaviors owing to individuals’ mutual interactions in various physical and sociological dynamical systems. This problem combines the mean-field theory with the LQ stochastic optimal control (see [1, 2]).
Lately, the mean-field problems have made many constructive and significative applications in various fields of mathematical finance, statistical mechanics, games theory (see [3]), especially in stochastic optimal control (see [4]). Some representative works in the mean-field optimal control, to name a few, include Li and Liu [5], Ma and Liu [6–8]. It is noteworthy that the optimal control problems of MF-LQ have received considerable attention. With regard to continuous-time cases, Yong [9] studied LQ optimal control problems for mean-field stochastic differential equations by variational method and decoupling technique; the same author in [10] systematically investigated the open-loop and closed-loop equilibrium solutions for the time-inconsistency MF-LQ optimal control problems. Subsequently, Huang et al. [11] generalized the results of Yong [9] to infinite horizon.
Nevertheless, discrete-time optimal control problems are more relevant to biomedical, engineering, economic, operation research and optimizing complex technological problems, etc. Recently, Elliott et al. [12] formulated the finite horizon discrete-time MF-LQ optimal control problem as an operator stochastic LQ optimal control problem. Later, the same authors in [13] discussed the infinite horizon case. Ni et al. [14] considered the indefinite mean-field stochastic LQ optimal control problem with finite horizon. Moreover, Song and Liu [15] derived the necessary and sufficient solvability condition of the finite horizon MF-LQ optimal control problem. Specially, here it is worth mentioning that Zhang et al. [16] presented the necessary and sufficient stabilization conditions of the MF-LQ optimal control problem subject to system (1.1). Nevertheless, the stabilization results in [16] mainly rely on a restrictive condition, namely
Indeed, it is a critical condition to study the MF-LQ optimal control problems. It is, therefore, natural to ask whether similar results can be derived if Q, Q̄, R, R̄ are just assumed to be symmetric, which is of particular and significant mathematical interest.
Inspired by the above arguments, in this paper we consider the following cost functional subject to system (1.1):
Here, the cost functional contains the explicitly correlative terms of state and control processes, namely \(G\neq 0\) and \(\bar{G}\neq 0\). More importantly, the weighting matrices Q, Q̄, R, R̄ are just symmetric, which is distinctly different from [16].
To the best of our knowledge, the study on the necessary and sufficient stabilization conditions of the discrete-time MF-LQ stochastic optimal control problems, especially with indefinite weighting matrices, is fairly scarce in the literature. Besides, the stabilization properties have not been investigated systematically. In this paper, we design the LQ optimal controller by means of the GAREs (see [17]), and we obtain the existence of the maximal solution to the original GAREs by introducing another one. Then, we show that the stabilizing solution is the maximal solution which is employed to present the optimal value. Furthermore, under the assumption of exact observability (see [18, 19]), we derive that the mean-field system is \(L^{2}\)-stabilizable if and only if the GAREs have a solution, which is also a maximal solution. Finally, we discuss the solvability of the GAREs by SDP method (see [20–23]), and we establish the relations among the GAREs, the SDP, and the MF-LQ optimal control problems.
The remainder of the paper is organized as follows. The next section gives the problem formulation and preliminaries. Section 3 is devoted to studying the GAREs. In Sect. 4, we discuss the solvability and stabilization for the MF-LQ optimal control problems. Section 5 establishes the relations among the GAREs, the SDP, and the MF-LQ optimal control problems. A couple of numerical examples are given in Sect. 6 to illustrate our main results. Section 7 gives some concluding remarks.
Most notations adopted in the paper are considerably standard as follows. \(A>0\)/\(A\geq 0\) means A is strictly positive definite/ positive semi-definite. \(A'\) denotes the transpose of any matrix or vector A. \(B^{-1}\) stands for the inverse of real matrix B. \(\dim (A)\)/\(\mathbf{R}(A)\)/\(\mathbf{Ker}(A)\) is the dimension/rank/kernel of A. \(\mathbb{T}^{n}\) represents the \(n \times n\) symmetric matrix. Denote by \(\mathcal{X}\) the space of all \(\mathbb{R}^{n}\)-valued square-integrable random variables. Let \(\mathbb{N}=\{0,1,\ldots ,N\}\), \(\tilde{\mathbb{N}}_{l}=\{l,l+1,\ldots \}\), \(\tilde{\mathbb{N}}_{0}=\tilde{\mathbb{N}}=\{0,1,2,\ldots \}\).
2 Problem formulation and preliminaries
In this paper, we study the infinite horizon MF-LQ optimal control problems. Indeed, to make the problems meaningful, the infinite horizon solution also requires to guarantee the closed-loop stability, which is interestingly different from the finite horizon cases. We firstly introduce the admissible control set
For simplicity, system (1.1) is denoted by \([A, \bar{A}, B, \bar{B}; C, \bar{C}, D, \bar{D}]\). In addition, \([A, \bar{A}; C, \bar{C}]\) denotes \([A, \bar{A}, 0, 0; C, \bar{C}, 0, 0]\), and \([A; C]\) denotes \([A, 0, 0, 0; C, 0, 0, 0]\).
The infinite horizon MF-LQ optimal control problem to be solved can be stated as follows.
Problem \(\mathcal {A}\)
For any \(\xi \in \mathcal{X}\), find \(u^{\ast } \in \mathscr{U}_{\infty }\) such that
where \(u^{\ast }\) is called an optimal control, and \(V(\cdot )\) is called the value function of Problem \(\mathcal{A}\).
Definition 2.1
For a matrix \(A\in \mathbb{R}^{n\times m}\), the Moore–Penrose inverse of A is defined to be the unique matrix \(A^{\dagger }\in \mathbb{R}^{m\times n}\) such that
- (i)
\(AA^{\dagger }A=A\), \(A^{\dagger }AA^{\dagger }=A^{\dagger }\);
- (ii)
\((AA^{\dagger })'=AA^{\dagger }\), \((A^{\dagger }A)'=A^{\dagger }A\).
Definition 2.2
System \([A, \bar{A}, B, \bar{B}; C, \bar{C}, D, \bar{D}]\) is called \(L^{2}\)-asymptotically stable if, for any \(\xi \in \mathcal{X}\), \(\lim_{k\to \infty } \mathbb{E}|x_{k}|^{2}=0\).
Definition 2.3
([24])
System \([A, \bar{A}, B, \bar{B}; C, \bar{C}, D, \bar{D}]\) is called closed-loop \(L^{2}\)-stabilizable if there exists a pair \((\mathcal{K}, \bar{\mathcal{K}}) \in \mathbb{R}^{m\times n}\times \mathbb{R}^{m\times n}\) such that, for any \(\xi \in \mathcal{X}\), the closed-loop system
is \(L^{2}\)-asymptotically stable. In this case, \(u_{k}=\mathcal{K}x_{k}+\bar{\mathcal{K}}\mathbb{E}x_{k} \) (\(k\in \tilde{\mathbb{N}}\)) is called the closed-loop \(L^{2}\)-stabilizer.
Definition 2.4
Consider the uncontrolled mean-field system
where and . System (2.1) (or \((A,\bar{A},C,\bar{C},\mathbb{Q}^{1/2})\), for short) is said to be exactly observable if, for any \(N\geq 0\),
Lemma 2.1
(Schur’s lemma)
Let matrices\(M=M'\), \(R=R'\), andNbe given with appropriate dimensions. Then the following statements are equivalent:
- (i)
\(M-NR^{-1}N'\geq \) (resp. >) 0.
- (ii)
(resp. >) 0.
- (iii)
(resp. >) 0.
In what follows, we make two assumptions.
- (A1):
\([A, \bar{A}, B, \bar{B}; C, \bar{C}, D, \bar{D}]\) is closed-loop \(L^{2}\)-stabilizable.
- (A2):
\((A, \bar{A}, C, \bar{C}, \mathbb{Q}^{1/2})\) is exactly observable.
We establish the following maximum principle which is the base to deriving the main results. Define
where \(P_{N+1}, \bar{P}_{N+1} \in \mathbb{T}^{n}\). The corresponding admissible control set is given as
Proposition 2.1
(Maximum principle)
The general maximum principle for minimizing (2.2) is presented as
where\(\beta _{k}\)satisfies
with the terminal condition
Proof
Denote
For any \(u_{k}\), \(\delta u_{k} \in \mathscr{U}_{N}\), \(\varepsilon \in (0,1)\), we have \(u_{k}^{\varepsilon }=u_{k}+\varepsilon \delta u_{k} \in \mathscr{U}_{N}\). By (2.8), we get that
Combining (2.6) with (2.7), for \(\delta x_{k}=x_{k}^{\varepsilon }-x_{k}\), it is obtained that
Seeing that \(\delta x_{0}=\delta \mathbb{E}x_{0}=0\), it yields that
where
Consequently,
where
Hence, the maximum principle can be written as
Furthermore, we shall show that (2.3)–(2.5) are equivalent to (2.9)–(2.10). Using (2.3)–(2.5), we can immediately get that the maximum principle can be reformulated as
Indeed, adding (2.13) to (2.11) and for \(k=N\), we have
which is exactly (2.9). Furthermore, by (2.12)–(2.13), we derive
Combining (2.3) with (2.14), it follows that
which is exactly (2.10). The proof is completed. □
Remark 2.1
Compared with most of previous works, the maximum principle for MF-LQ optimal control problem was based on the mean-field backward stochastic differential equation (see [25, 26]), while Proposition 2.1 provides a convenient calculation method and can be reduced to the standard stochastic LQ case.
3 GAREs
In this section, we present several results about the GAREs which play a key role in deriving our main results. Now, we introduce the following GAREs:
where
Definition 3.1
GAREs (3.1) are said to be solvable if \(\rho _{k}^{(i)}[\rho _{k}^{(i)}]^{\dagger }M_{k}^{(i)}=M_{k}^{(i)}\), \(i=1,2\), are satisfied for \(k\in \mathbb{N}\).
Motivated by [27] of Ait Rami et al. (2002), we denote
Here, \(\Delta ^{(1)}=\mathbf{Ker}B\cap \mathbf{Ker}D\), \(\Delta ^{(2)}=\mathbf{Ker}(B+\bar{B})\cap \mathbf{Ker}(D+\bar{D})\), and
Theorem 3.1
If\(\boldsymbol{\varGamma }\neq \emptyset \), then for any terminal value\((P_{N+1},\bar{P}_{N+1})=(\hat{P},\check{P})\in \boldsymbol{\varGamma }\), GAREs (3.1) are solvable and converge to the following GAREs:
where\(\rho ^{(i)}\), \(M^{(i)}\)are given by
Proof
For any \((\hat{P},\check{P})\in \boldsymbol{\varGamma }\), define the new GAREs (NGAREs)
where \(T_{N+1}=\bar{T}_{N+1}=0\) and
The corresponding new cost functional is given by
To make the time horizon N specific in the finite horizon MF-LQ problem, we denote \(T_{k}\), \(\bar{T}_{k}\) in (3.3) as \(T_{k}(N)\), \(\bar{T}_{k}(N)\). Set
where \(x_{l}=\xi \). According to (3.5), for \(l_{1}< l_{2}\), we get
Seeing the time-invariance of the coefficient matrices, it is obtained that \(T_{l}(N)=T_{0}(N-l)\), \(\bar{T}_{l}(N)=\bar{T}_{0}(N-l)\). Combining (3.5) with (3.6), for any \(x_{0}=\xi \in \mathcal{X}\) and any stabilizing controller \(u_{k}=\mathcal{L}x_{k}+\bar{\mathcal{L}}\mathbb{E}x_{k}\), it follows that
where \(c>0\) is a constant. Selecting \(x_{0}\in \mathbb{R}^{n}\), we claim that, for any N, l,
Meanwhile, let \(x_{0}=\varphi \delta \) with \(\varphi \in \mathbb{R}^{n}\) and \(P(\delta =-1)=P(\delta =1)=\frac{1}{2}\). By virtue of (3.8), we obtain
From (3.7) and (3.9)–(3.10), we get
Here, \(C_{1}\), \(\bar{C}_{1}\) are bounded. Taking \(l\rightarrow -\infty \) and letting \(P_{k}=T_{k}+\hat{P}\), \(\bar{P}_{k}=\bar{T}_{k}+\check{P}\), then \(P_{k}\), \(\bar{P}_{k}\) increase with respect to k, and \((P_{k},\bar{P_{k}})\) converges to \((P,\bar{P})\) with \((P,\bar{P})\) satisfying GAREs (3.2). □
Remark 3.1
In view of the regular conditions in (3.3), we have
where \(\mathcal{L}_{k}\), \(\bar{\mathcal{L}}_{k}\) satisfy
Besides,
where
Proposition 3.1
Under\(\boldsymbol{\varGamma }\neq \emptyset \), for NGAREs (3.3) with the terminal conditions\(T_{N+1}(N)=\bar{T}_{N+1}(N)=0\), assume that\(\alpha _{k}^{(i)}[\alpha _{k}^{(i)}]^{\dagger }H_{k}^{(i)}=H_{k}^{(i)}\), \(i=1,2\). Then the optimal controller is designed as\(u_{k}^{\star }=\mathcal{L}_{k}x_{k}+\bar{\mathcal{L}}_{k}\mathbb{E}x_{k}\), where\(\mathcal{L}_{k}\), \(\bar{\mathcal{L}}_{k}\)are given as (3.11), and the optimal value is derived by
Proof
Denote
let \(k=N\), by \(\boldsymbol{\varGamma }\neq \emptyset \), it yields that
Indeed, if \(\mathbb{E}u_{N}=0\), \(u_{N}\neq 0\), then \(\bar{J}(N)=\mathbb{E}[u_{N}'\alpha _{N}^{(1)}u_{N}]>0\); further, we have \(\alpha _{N}^{(1)}>0\). Similarly, if \(u_{N}=\mathbb{E}u_{N}\neq 0\), we have \(\alpha _{N}^{(2)}>0\). Using the maximum principle in Proposition 2.1, it follows that
Adding from \(k=l+1\) to \(k=N\) on both sides of the above equation, we obtain
Furthermore,
Now we prove \(\alpha _{l}^{(1)}>0\) and \(\alpha _{l}^{(2)}>0\). Let \(x_{l}=0\),
then for any \(u_{l}\neq 0\), \(\bar{J}(l)>0\). Following the discussion of (3.12), it implies that \(\alpha _{l}^{(1)}>0\) and \(\alpha _{l}^{(2)}>0\). Namely, \(\alpha _{k}^{(1)}\geq 0\), \(\alpha _{k}^{(2)}\geq 0\) for \(k\geq 0\). Furthermore,
Therefore, \(\bar{J}_{T}^{\ast }(0,\xi ;u)=\mathbb{E}[x_{0}'T_{0}(N)x_{0}]+( \mathbb{E}x_{0})'\bar{T}_{0}(N)\mathbb{E}x_{0}\). The proof is completed. □
Generally speaking, the uniqueness of the solution to the GAREs is not guaranteed. Next, we shall focus on the properties of the maximal solution and its relation with the stabilizing solution.
Definition 3.2
A solution of GAREs (3.2) is called the maximal solution, denoted by \((P^{\ast }, \bar{P}^{\ast })\), if for any solution \((P, \bar{P})\) of GAREs (3.2), \(P^{\ast } \geq P\), \(\bar{P}^{\ast } \geq \bar{P}\) hold.
By Definition 3.2, it is clear that the maximal solution must be unique if it exists.
Definition 3.3
([13])
A solution \((P, \bar{P})\) of GAREs (3.2) is called a stabilizing solution if the closed-loop control
stabilizes the mean-field system (1.1).
Now, we introduce a compact form of GAREs (3.2). Let , then
where
with
By considering GARE (3.13), we can immediately show the existence of the maximal solution to GAREs (3.2).
Proposition 3.2
Suppose that GAREs (3.2) and GARE (3.13) have the maximal solutions and the stabilizing solutions, then the following statements hold.
- (i)
\(\tilde{P}^{\ast }=\operatorname{diag} \{ P^{\ast }+{\bar{P}}^{\ast }, {P}^{ \ast } \} \)is the maximal solution to GARE (3.13) if and only if\(({P}^{\ast }, {\bar{P}}^{\ast })\)is the maximal solution to GAREs (3.2).
- (ii)
\(\tilde{P}=\operatorname{diag} \{ P+\bar{P}, P \} \)is a stabilizing solution to GARE (3.13) if and only if\((P, \bar{P})\)is the stabilizing solution to GAREs (3.2).
Proof
(i) It is clear.
(ii) Seeing that and resorting to the method in [13, Lemma 5.1], \(\{\nu _{k}=-\rho ^{\dagger }M, k\in \tilde{\mathbb{N}}\}\) is a stabilizing control of the following system:
where \(\varphi =\{\varphi _{k}, k\in \tilde{\mathbb{N}}\}\) and \(\theta =\{\theta _{k}, k\in \tilde{\mathbb{N}}\}\) are two mutually independent martingale difference sequences if and only if \(\{u_{k}=-[\rho ^{(2)}]^{\dagger }M^{(2)}\mathbb{E}x_{k}-[\rho ^{(1)}]^{ \dagger }M^{(1)}(x_{k}-\mathbb{E}x_{k}), k\in \tilde{\mathbb{N}}\}\) is a stabilizing control of (1.1). Thus, the conclusion holds. □
Proposition 3.3
If\(\boldsymbol{\varGamma }\neq \emptyset \)and (A1) are satisfied, then a stabilizing solution to GARE (3.13) is the maximal solution.
Proof
Let \(z=( z_{1}' \ z_{2}' \ z_{3}' \ z_{4}' )'\in \mathbb{R}^{2(n+m)}\), \(z_{1}, z_{3}\in \mathbb{R}^{n}\), \(z_{2}, z_{4}\in \mathbb{R}^{m}\), we get that
thus, it immediately yields that
Then, by the method in [28, Theorem 5.3.1] of Damn (2004), and [29] of Abou-Kandil et al. (2003). This statement is standard due to the space limitations, here we omit the proof. □
Corollary 3.1
If\(\boldsymbol{\varGamma }\neq \emptyset \)and (A1) are satisfied, then the following statements hold.
- (i)
GAREs (3.2) admit at most one stabilizing solution.
- (ii)
If there is\((P_{\star }, P_{\star }+\bar{P}_{\star })\)such that, \(i=1,2\), then GAREs (3.2) admit a stabilizing solution.
Proof
(i) Combining Propositions 3.2 with 3.3 and noticing the uniqueness of the maximal solution, we could immediately get the result.
(ii) We could resort to the method in [30, Theorem 4.3] of Ait Rami et al. (2001). Owing to the space limitations, here we omit the proof. □
4 MF-LQ optimal control
Now, we return to the infinite horizon stochastic MF-LQ optimal control problems.
Theorem 4.1
If\(\boldsymbol{\varGamma } \neq \emptyset \)and (A1) are satisfied, then Problem\(\mathcal{A}\)has an optimal control
and the optimal value is given by
Here, \((P^{\ast }, \bar{P}^{\ast })\)is the maximal solution to GAREs (3.2), and\((\rho ^{(i)})^{\ast }\), \((M^{(i)})^{\ast }\) (\(i=1,2\)) are those in Theorem 3.1with\((P, \bar{P})\)replaced by\((P^{\ast }, \bar{P}^{\ast })\).
Proof
By means of completing the squares, it yields that
Since \(\rho ^{(1)}\), \(\rho ^{(2)}\geq 0\), we can immediately get that \(J(\xi ,u)\geq \mathbb{E}(x_{0}'P^{\ast }x_{0})+(\mathbb{E}x_{0})' \bar{P}^{\ast }\mathbb{E}x_{0}\). Thus,
Next, we shall show \(V(x_{0})=\inf_{u\in \mathscr{U}_{\infty }}J(\xi ,u)\leq \mathbb{E}(x_{0}'P^{ \ast }x_{0})+(\mathbb{E}x_{0})'\bar{P}^{\ast }\mathbb{E}x_{0}\) by the standard perturbation method. For a positive decreasing sequence \(\{\varepsilon _{i}, i=0, 1, 2,\ldots \}\), we consider the GAREs with Q, R replaced by \(Q+\varepsilon _{i}I\), \(R+\varepsilon _{i}I\), respectively. And the corresponding stabilizing solution to the GAREs is written as \((P_{\varepsilon _{i}}, \bar{P}_{\varepsilon _{i}})\), which is the maximal solution. Indeed, \(P_{\varepsilon _{0}}\geq \cdots \geq P_{\varepsilon _{i}}\geq P_{ \varepsilon _{i+1}}\geq \cdots \geq P\), \(\bar{P}_{\varepsilon _{0}} \geq \cdots \geq \bar{P}_{\varepsilon _{i}}\geq \bar{P}_{\varepsilon _{i+1}} \geq \cdots \geq \bar{P}\). Moreover, \(\lim_{i\to \infty }P_{ \varepsilon _{i}}=P\), \(\lim_{i\to \infty }\bar{P}_{\varepsilon _{i}}= \bar{P}\). Set \(u_{k}^{i}=-[\rho _{i}^{(2)}]^{\dagger }M_{i}^{(2)}\mathbb{E}x_{k} -[ \rho _{i}^{(1)}]^{\dagger }M_{i}^{(1)}(x_{k}-\mathbb{E}x_{k})\), then
Let \(i\to \infty \), we have
Combining (4.3) with (4.4), we have \(V(x_{0})=\mathbb{E}(x_{0}'P^{\ast }x_{0})+(\mathbb{E}x_{0})'\bar{P}^{ \ast }\mathbb{E}x_{0}\). Meanwhile, the optimal controller \(u_{k}\) satisfies
Thus, \(u_{k}=-[(\rho ^{(2)})^{\ast }]^{\dagger }[M^{(2)}]^{\ast }\mathbb{E}x_{k} -[(\rho ^{(1)})^{\ast }]^{\dagger }[M^{(1)}]^{\ast }(x_{k}-\mathbb{E}x_{k})\). □
Remark 4.1
In Theorem 4.1, we show that the optimal value of Problem \(\mathcal{A}\) can be presented by virtue of the maximal solution to GAREs (3.2).
A classical problem in optimal control theory is the so-called LQ stabilization problem. Now, we consider the stabilization of the indefinite discrete-time MF-LQ control problem.
Problem \(\mathcal {B}\)
Find a \(\mathfrak{F}_{k}\)-measurable \(u_{k}\) to minimize the cost functional (1.2), simultaneously, to stabilize the mean-field system (1.1).
Definition 4.1
NGAREs (3.3) are said to have a positive semi-definite (definite) solution if they admit \(T\geq 0\), \(T+\bar{T}\geq 0\) (\(T>0\), \(T+\bar{T}> 0\)) satisfying NGAREs (3.3).
Theorem 4.2
If\(\bar{\boldsymbol{\varGamma }}\neq \emptyset \)and (A2) are satisfied, then system (1.1) is\(L^{2}\)-stabilizable if and only if there exists a solution to GAREs (3.2), which is also the maximal solution. In this case, the optimal stabilizing solution and the optimal value can be designed as (4.1)–(4.2), respectively.
Proof
Necessity: Under (A2) and \(\bar{\boldsymbol{\varGamma }}\neq \emptyset \), assume that system (1.1) is \(L^{2}\)-stabilizable, we shall show that GAREs (3.2) have a solution \((P,\bar{P})\), which is also a maximal solution. Notice that Remark 3.1 and \(T_{N+1}(N)=\bar{T}_{N+1}(N)=0\), by virtue of the induction, we can immediately get that \(T_{k}(N)\geq 0\), \(\bar{T}_{k}(N)\geq 0\) for \(0\leq k\leq N\).
Step 1. We shall state that \(\alpha _{k}^{(i)}[\alpha _{k}^{(i)}]^{\dagger }H_{k}^{(i)}=H_{k}^{(i)}\), \(i=1,2\). Since \(\alpha _{k}^{(1)}\geq 0\), we obtain that
in which \(( U_{k}^{1} \ U_{k}^{2} )\) is an orthogonal matrix, \(V_{k}>0\) and \(\dim (V_{k})= \mathbf{R}(\alpha _{k}^{(1)})\), and the columns of the matrix \(U_{k}^{2}\) form a basis of \(\mathbf{Ker}(\alpha _{k}^{(1)})\). Combining \(\lambda ^{(1)}\geq 0\) with (3.4), we derive \(\mathbf{Ker}(\alpha _{k}^{(1)}) \in \mathbf{Ker}(\lambda ^{(1)})\). Moreover, in view of \(\alpha _{k}^{(1)}U_{k}^{2}(U_{k}^{2})'=0\), then \(\lambda ^{(1)}U_{k}^{2}(U_{k}^{2})'=0\). We can easily calculate
Meanwhile, since \(\mathbf{Ker}(\Delta ^{(1)})\subseteq (\mathbf{Ker}B\cap \mathbf{Ker}D)\), we deduce that \(\alpha _{k}^{(1)}[\alpha _{k}^{(1)}]^{\dagger }H_{k}^{(1)}=H_{k}^{(1)}\). Similarly, \(\alpha _{k}^{(2)}[\alpha _{k}^{(2)}]^{\dagger }H_{k}^{(2)}=H_{k}^{(2)}\) follows.
Step 2. We shall prove that \(T_{k}(N)\), \(\bar{T}_{k}(N)\) are convergent. Using Proposition 3.1, it is clear that the optimal controller and optimal cost value of (3.5) subject to (1.1) are, respectively, designed by
Accordingly, for any N, it yields that
Let \(x_{0}\neq 0\), \(\mathbb{E}x_{0}=0\), then \(T_{0}(N)\leq T_{0}(N+1)\). Similarly, let \(x_{0}\neq 0\), \(\mathbb{E}x_{0}=x_{0}\), we have \(T_{0}(N)+\bar{T}_{0}(N)\leq T_{0}(N+1)+\bar{T}_{0}(N+1)\). Namely, \(T_{0}(N)\), \(T_{0}(N)+\bar{T}_{0}(N)\) increase with respect to N.
Next, we state that \(T_{0}(N)\), \(T_{0}(N)+\bar{T}_{0}(N)\) are bounded. Since system (1.1) is \(L^{2}\)-stabilizable, there exists \(u_{k}=\mathcal{L}x_{k}+\bar{\mathcal{L}}\mathbb{E}x_{k}\) such that system (1.1) satisfies \(\lim_{k\to +\infty }\mathbb{E}(x_{k}'x_{k})=0\). Notice that
thus, \(\lim_{k\to +\infty }(\mathbb{E}x_{k})'\mathbb{E}x_{k}=0\). Resorting to [30, Lemma 4.1], we obtain \(\mathbb{E}\sum_{k=0}^{\infty }(x_{k}'x_{k})< +\infty \). Using (4.6), we get \(\sum_{k=0}^{\infty }(\mathbb{E}x_{k})'\mathbb{E}x_{k}< +\infty \). On the other hand, noticing \(\bar{\boldsymbol{\varGamma }}\neq \emptyset \), there exists a constant b such that , . Hence, we claim that
By (4.5), we have
Let \(\mathbb{E}x_{0}=0\), we have \(0\leq T_{0}(N)\leq b c I\). Similarly, let \(x_{0}=\mathbb{E}x_{0}\), we get \(0\leq T_{0}(N)+\bar{T}_{0}(N)\leq b c I\). Consequently, \(T_{0}(N)\) and \(T_{0}(N)+\bar{T}_{0}(N)\) are bounded. Furthermore, \(T_{0}(N)\) and \(T_{0}(N)+\bar{T}_{0}(N)\) are convergent. Observe the time-invariance, we derive
In this case, \(\lim_{N\rightarrow \infty }\alpha _{k}^{(i)}=\alpha ^{(i)}\), \(\lim_{N \rightarrow \infty }H_{k}^{(i)}=H^{(i)}\), \(i=1,2\). Consequently, we deduce that \((T,\bar{T})\) is the solution to the following NGAREs:
where
Step 3. We shall prove \(T>0\), \(\bar{T}>0\). In view of \(T_{k}(N)\geq 0\), \(\bar{T}_{k}(N)\geq 0\), we get that \(T\geq 0\), \(\bar{T}\geq 0\). Specifically, suppose it does not hold. Seeing that \(\mathbb{E}(x_{0}'x_{0})=\mathbb{E}(\mathbb{X}_{0}'\mathbb{X}_{0})\), then there is \(\mathbb{X}_{0}\neq 0\) such that \(\mathbb{E}(\mathbb{X}_{0}'\tilde{T}\mathbb{X}_{0})=0\) with \(\tilde{T}=\operatorname{diag}(T,T+\bar{T})\). Define the Lyapunov function candidate as
Using (4.7), it follows that
where
According to Remark 3.1, we have \(\mathcal{Q}\geq 0\), \(\bar{\mathcal{Q}}\geq 0\); further, it is obtained that
Adding from 0 to N on both sides of (4.8), then
Consequently,
Resorting to the method of [31, Theorem 4, Proposition 1], we get that the exact observability of system \((A, \bar{A}, C, \bar{C}, \mathbb{Q}^{1/2})\) is equivalent to the exact observability of the following system:
Here,
Thus, by (4.9), we get \(\mathbb{X}_{0}=0\), which is contradiction. In summary, \(T>0\) and \(T+\bar{T}>0\) hold.
Step 4. From Theorem 3.1, we see \(P_{k}(N)=T_{k}(N)+\hat{P}\), \(\bar{P}_{k}(N)=\bar{T}_{k}(N)+\check{P}\). Recall the convergences of \(P_{k}(N)\), \(\bar{P}_{k}(N)\), then \(P=T+\hat{P}\), \(\bar{P}=\bar{T}+\check{P}\). Besides, combining the arbitrariness of P̂, P̌ with \(T, \bar{T}>0\), we derive \(P\geq \hat{P}\), \(\bar{P}\geq \check{P}\), namely \((P,\bar{P})\) is the maximal solution to GAREs (3.2).
Sufficiency. Under (A2) and \(\bar{\boldsymbol{\varGamma }}\neq \emptyset \), assume that GAREs (3.2) have a solution, we shall prove that system (1.1) is \(L^{2}\)-stabilizable. Following the proof of the necessity, we claim that if GAREs (3.2) have a solution \((P,\bar{P})\), then NGAREs (4.7) have a positive definite solution \((T,\bar{T})\). In addition, \(P=T+\hat{P}\), \(\bar{P}=\bar{T}+\check{P}\). Notice that \(\mathcal{K}=\mathcal{L}\), \(\bar{\mathcal{K}}=\bar{\mathcal{L}}\), the stabilization of system (1.1) with \(u_{k}=\mathcal{K}x_{k}+\bar{\mathcal{K}}\mathbb{E}x_{k}\) is equivalent to the stabilization of system (1.1) with \(u_{k}=\mathcal{L}x_{k}+\bar{\mathcal{L}}\mathbb{E}x_{k}\). Together with Remark 3.1, (4.8) can be reformulated as
which implies that \(V_{T}(k,x_{k})\) is decreasing with respect to k. Along with \(V_{T}(k,x_{k})\geq 0\), we can deduce that \(V_{T}(k,x_{k})\) is convergent. Adding from m to \(m+N\) on both sides of the above equation and taking limitation, we get
By a time-shift, it yields that
We further obtain
hence, \(\lim_{m\to +\infty } \mathbb{E}(x_{m}'x_{m})=0\). Namely, system (1.1) is \(L^{2}\)-stabilizable. Using Theorem 4.1, the optimal controller and optimal value can be designed as (4.1)–(4.2), respectively. □
Remark 4.2
Our results extend and improve the ones in [16]. Besides, Theorem 4.2 makes it clear that the solvability of GAREs (3.2) with indefinite weighting matrices is equivalent to the solvability of NGAREs (4.7) with positive semi-definite weighting matrices. Simultaneously, it also indicates that the stabilization problems with indefinite weighting matrices can be reduced to a positive semi-definite case. So to speak, these conclusions will give us fresh ideas to consider the indefinite MF-LQ optimal control problems, especially to consider their stabilization problems.
Remark 4.3
Theorem 4.2 presents the necessary and sufficient stabilization condition for the indefinite MF-LQ optimal control problem, while for most of previous works, stabilization was the precondition for the indefinite control problems. In other words, their conclusions were only to discuss the existence of the stabilizing solution to the GAREs based on the assumption of the stabilization.
5 Characterizing MF-LQ problem via SDP
In this section, we present some results with respect to the SDP problem. Meanwhile, we establish the relations among the GAREs, the SDP, and the MF-LQ optimal control problems.
Definition 5.1
Let a vector \(a=(a_{1}, a_{2}, \ldots , a_{m})'\in \mathbb{R}^{m}\) and matrices \(F_{0}, F_{1}, \ldots , F_{m} \in \mathbb{T}^{n}\) be given. The following optimization problem:
is called a SDP. Besides, the dual problem of SDP (5.1) is defined as
Specifically, we consider the following SDP problem:
Theorem 5.1
A unique optimal solution is admitted to SDP problem (5.2), which is also the maximal solution to GAREs (3.2).
Proof
Let \((P_{\ast },\bar{P}_{\ast })\) be an optimal solution to SDP problem (5.2). In order to show that it is indeed a maximal solution, denote
By a simple calculation, we have
On the other hand, \(u_{\ast }=\mathcal{K}_{1}x_{k}+(\mathcal{K}_{2}-\mathcal{K}_{1}) \mathbb{E}x_{k}\) is a stabilizing control. Following the proof of [11, Theorem 6.7], it follows that \((P_{\ast },\bar{P}_{\ast })\) is the upper bound of the set Γ; in other words, \((P_{\ast },\bar{P}_{\ast })\) is the maximal solution. Furthermore, the uniqueness of the solution to SDP (5.2) follows from the maximality. The proof is completed. □
Corollary 5.1
The following statements are equivalent: (i) \(\boldsymbol{\varGamma }\neq \emptyset \); (ii) There is a solution to GAREs (3.2).
Besides, while either (i) or (ii) holds, GAREs (3.2) have a maximal solution\((P^{\ast },\bar{P}^{\ast })\), which is the unique optimal solution to SDP problem (5.2).
Corollary 5.2
Let (A1) and\(\boldsymbol{\varGamma } \neq \emptyset \)hold, then Problem\(\mathcal{B}\)admits an optimal control\(u_{k}=\mathcal{K}^{\ast }x_{k}+\bar{\mathcal{K}}^{\ast }\mathbb{E}x_{k}\), \(k \in \tilde{\mathbb{N}}\), and the optimal value is given as\(V(x_{0})=\mathbb{E}(x_{0}'P^{\ast }x_{0})+(\mathbb{E}x_{0})'\bar{P}^{ \ast }\mathbb{E}x_{0}\). Here, \((P^{\ast }, \bar{P}^{\ast })\)is the maximal solution to GAREs (3.2), which is the unique optimal solution to SDP problem (5.2).
6 Numerical results
In this section, we give some numerical examples to illustrate our main results.
Example 6.1
Consider system (1.1) and cost functional (1.2) with
After running the calculation of the SDP theory via Matlab software, we obtain that
Furthermore, we have
It is clear that \(\mathbf{Ker}(\lambda ^{(1)})\subseteq \mathbf{Ker}B\cap \mathbf{Ker}D\), \(\mathbf{Ker}(\lambda ^{(2)})\subseteq \mathbf{Ker}(B+\bar{B})\cap \mathbf{Ker}(D+\bar{D})\). Using Lemma 2.1, it is easy to see , \(i=1,2\). In other words, \(\bar{\boldsymbol{\varGamma }}\neq \emptyset \). Thus, by Theorem 4.2, we deduce that system (1.1) is \(L^{2}\)-stabilizable, and the optimal controller is designed by
where
A curve of \(\mathbb{E}|x_{k}|^{2}\) under control (6.1) is shown in Fig. 1. As expected, the curve is convergent.
Example 6.2
Consider system (1.1) and cost functional (1.2) with
By virtue of SDP theory, we have
Furthermore,
Similar to Example 6.1, it is easy to verify \(\bar{\boldsymbol{\varGamma }}=\emptyset \). In this case,
According to Theorem 4.2, we know that system (1.1) is not \(L^{2}\)-stabilizable. A curve of \(\mathbb{E}|x_{k}|^{2}\) under control (6.1) is shown in Fig. 2. As expected, the curve is not convergent.
To further illustrate Theorem 4.2, we give the following two examples as well.
Example 6.3
Consider system (1.1) and cost functional (1.2) with
Likewise, by virtue of the SDP theory, we deduce that \(P^{\ast }=1.5625\), \(\bar{P}^{\ast }=0.7722\). Then
Since \(\bar{\boldsymbol{\varGamma }}\neq \emptyset \), by Theorem 4.2, there is a unique optimal controller to stabilize system (1.1); meanwhile, to minimize cost functional (1.2), the optimal controller is given as \(u_{k}=-1.3333x_{k}-0.1305\mathbb{E}x_{k}\), \(k\geq 0\). According to the optimal controller, the simulation of the system state and the curve of \(\mathbb{E}|x_{k}|^{2}\) are shown in Fig. 3. As expected, the curve is convergent.
Example 6.4
Consider system (1.1) and cost functional (1.2) with
By GAREs (3.2), we get that P has two different negative roots \(P=-0.1604\), \(P=-0.5670\), and P̄ has a negative root \(\bar{P}=-0.1548\). Then we have the following two sets of solutions:
In the above two cases, \(\bar{\boldsymbol{\varGamma }}=\emptyset \). Furthermore, when \(P=-0.1604\), we can get \(\mathcal{K}=-0.1027\), \(\bar{\mathcal{K}}=-0.5975\). Similarly, when \(P=-0.5670\), we get \(\mathcal{K}=-1.2863\), \(\bar{\mathcal{K}}=-19.0322\). Thus, the controllers are presented as \(u_{k}=-0.1027x_{k}-0.5975\mathbb{E}x_{k}\), \(u_{k}=-1.2863x_{k}-19.0322 \mathbb{E}x_{k}\), respectively. Simulation results for the curve of \(\mathbb{E}|x_{k}|^{2}\) with the corresponding optimal controller are shown as in Fig. 4 and Fig. 5, respectively. As expected, the curves are not convergent.
7 Concluding remarks
We have investigated the exact observability of a linear stochastic time-invariant system in this work. How to extend various definitions to the linear stochastic time-varying system is a meaningful topic that merits further discussions. Compared with the time-invariant system, defining the exact observability for the time-varying stochastic system is much more difficult and sophisticated. In addition, the necessary and sufficient stabilization conditions also deserve to be systematically studied. Thus, we attempt to discuss the linear time-varying system deeply in the future.
References
Ait Rami, M., Zhou, X.Y.: Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Autom. Control 45(6), 1131–1143 (2000)
Chen, S., Li, X., Zhou, X.: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36(5), 1685–1702 (1998)
Lasry, J.M., Lions, P.L.: Mean field games. Jpn. J. Math. 2(1), 229–260 (2007)
Yong, J.M., Zhou, X.Y.: Stochastic Controls, Hamiltonian Systems and HJB Equations. Springer, Berlin (1999)
Li, R.J., Liu, B.: A maximum principle for fully coupled stochastic control systems of mean-field type. J. Math. Anal. Appl. 415(2), 902–930 (2014)
Ma, H.P., Liu, B.: Maximum principle for partially observed risk-sensitive optimal control problems of mean-field type. Eur. J. Control 32, 16–23 (2016)
Ma, H.P., Liu, B.: Infinite horizon optimal control problem of mean-field backward stochastic delay differential equation under partial information. Eur. J. Control 36, 43–50 (2017)
Ma, H.P., Liu, B.: Optimal control of mean-field jump-diffusion systems with noisy memory. Int. J. Control 92(4), 816–827 (2019)
Yong, J.M.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51(4), 2809–2838 (2013)
Yong, J.M.: Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions. Trans. Am. Math. Soc. 369(8), 5467–5523 (2017)
Huang, J.H., Li, X., Yong, J.M.: A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Relat. Fields 5(1), 97–139 (2015)
Elliott, R., Li, X., Ni, Y.H.: Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica 49(11), 3222–3233 (2013)
Ni, Y.H., Elliott, R., Li, X.: Discrete-time mean-field stochastic linear-quadratic optimal control problems, II: infinite horizon case. Automatica 57, 65–77 (2015)
Ni, Y.H., Zhang, J.F., Li, X.: Indefinite mean-field stochastic linear-quadratic optimal control. IEEE Trans. Autom. Control 60(7), 1786–1800 (2015)
Song, T., Liu, B.: Discrete-time stochastic mean-field linear quadratic optimal control problem with finite horizon. Asian J. Control 1–11 (2020). https://doi.org/10.1002/asjc.2306
Zhang, H.S., Qi, Q.Y., Fu, M.Y.: Optimal stabilization control for discrete-time mean-field stochastic systems. IEEE Trans. Autom. Control 64(3), 1125–1136 (2019)
Lancaster, P., Rodman, L.: Algebraic Riccati Equations. Oxford University Press, New York (1995)
Anderson, B.D.O., Moore, J.B.: Detectability and observability of time-varying discrete-time systems. SIAM J. Control Optim. 19(1), 20–32 (1981)
Li, Z.Y., Wang, Y., Zhou, B., et al.: Detectability and observability of discrete-time stochastic systems and their applications. Automatica 45(5), 1340–1346 (2009)
Ait Rami, M., Chen, X., Moore, J.B., et al.: Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim. 40(4), 1296–1311 (2001)
Ait Rami, M., El Ghaoui, L.: LMI optimization for nonstandard Riccati equations arising in stochastic control. IEEE Trans. Autom. Control 41(11), 1666–1671 (1996)
Ait Rami, M., Zhou, X.Y., Moore, J.B.: Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon. Syst. Control Lett. 41(2), 123–133 (2000)
Yao, D., Zhang, S.Z., Zhou, X.Y.: Stochastic linear-quadratic control via primal-dual semidefinite programming. SIAM Rev. 46, 87–111 (2004)
Ni, Y.H., Li, X., Zhang, J.F.: Indefinite mean-field stochastic linear-quadratic optimal control: from finite horizon to infinite horizon. IEEE Trans. Autom. Control 61(11), 3269–3284 (2016)
Buckdahn, R., Djehiche, B., Li, J., et al.: Mean-field backward stochastic differential equations: a limit approach. Ann. Probab. 37(4), 1524–1565 (2009)
Li, J.: Stochastic maximum principle in the mean-field controls. Automatica 48(2), 366–373 (2012)
Ait Rami, M., Chen, X., Zhou, X.Y.: Discrete-time indefinite LQ control with state and control dependent noises. J. Glob. Optim. 23(3–4), 245–265 (2002)
Damm, T.: Rational Matrix Equations in Stochastic Control. Lecture Notes in Control and Information Sciences, vol. 297. Springer, New York (2004)
Abou-Kandil, H., Freiling, G., Ionescu, V., et al.: Matrix Riccati Equations in Control and Systems Theory. Birkhäuser, Basel (2003)
Ait Rami, M., Chen, X., Moore, J.B., et al.: Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls. IEEE Trans. Autom. Control 46(3), 428–440 (2001)
Zhang, W.H., Chen, B.S.: On stabilizability and exact observability of stochastic systems with their applications. Automatica 40(1), 87–94 (2004)
Acknowledgements
The authors would like to thank the anonymous referees very much for helpful comments and suggestions which led to the improvement of presentation and quality of the work.
Availability of data and materials
Not applicable.
Funding
This work is partially supported by the National Natural Science Foundation of China (Grant No. 11971185).
Author information
Authors and Affiliations
Contributions
All authors contributed equally and significantly in this manuscript and they read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Song, T., Liu, B. Solvability and optimal stabilization controls of discrete-time mean-field stochastic system with infinite horizon. Adv Differ Equ 2020, 187 (2020). https://doi.org/10.1186/s13662-020-02639-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13662-020-02639-4
MSC
- 49N10
- 93E20
- 93C55
Keywords
- Indefinite stochastic LQ control
- Discrete-time system
- Mean-field theory
- Stabilization
- Generalized algebraic Riccati equation