 Research
 Open access
 Published:
On the maximum principle for relaxed control problems of nonlinear stochastic systems
Advances in Continuous and Discrete Models volume 2024, Article number: 8 (2024)
Abstract
We consider optimal control problems for a system governed by a stochastic differential equation driven by a ddimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measurevalued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the socalled chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng’s maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable.
1 Introduction
Our main goal in this paper is to prove a stochastic maximum principle for relaxed controls in the case where both the drift and the diffusion coefficient are controlled.
It is well known that the two main approaches to handling optimal control problems are the dynamic programming by Bellman and the maximum principle by Pontryagin [36]. The maximum principle provides a set of necessary conditions for optimality that an optimal control must satisfy, as detailed in [36]. These conditions include a forward equation for the state process, a backward equation for the adjoint variable, and minimization of the Hamiltonian function.
Within stochastic control, two main approaches for a stochastic maximum principle emerge, based on the solution concept (weak/strong) and control type (openloop/feedback). The first approach, for strong solutions with openloop controls, was established by [23] using spike variations. In [17] the author employed martingale methods and the Girsanov theorem to derive a maximum principle for weak solutions with feedback controls.
In [10] the author addressed systems where the diffusion coefficient depends on the control variable, utilizing convex perturbations and the firstorder adjoint variable. His result constitutes a weak maximum principle with the variational inequality applied to the Gâteaux derivative of the Hamiltonian. In [30], a global maximum principle was established for a nonconvex domain and controlled diffusion coefficient, involving the introduction of a secondorder adjoint process. This extension was further developed for jumpdiffusion processes in [33].
Pontryagin’s maximum principle has proven widespread application in solving problems related to mathematical finance and portfolio optimization, as shown in [31]. We recommend [36] for a comprehensive overview and detailed references on the subject.
The main motivation behind relaxed controls lies in their property to guarantee the existence of optimal solutions in this class. This concept originated with Young’s work [37] on generalized solutions in the calculus of variations, leading to the notion of Young measure. Subsequently, this framework was extended to deterministic control theory, giving rise to the concept of relaxed control. A key challenge in nonconvex control problems arises from the lack of closure of the set of strict controls under simple convergence of measurable functions. Relaxed controls elegantly address this challenge by replacing strict controls with random Dirac probability measures. This effectively transforms the set of strict controls into a compact subset of probability measures, ensuring closure under the topology of weak convergence. This “relaxation” of the convergence requirement enables us to formulate the optimal control problem as a continuous function optimization over a compact metric space, guaranteeing the existence of an optimal solution. The authors in [9, 16, 24] established the first existence results of relaxed controls for stochastic differential equations with uncontrolled diffusion coefficients. Subsequently, more complex systems with controlled diffusion coefficients were tackled by [14, 18, 19]. Using Krylov’s method of Markovian selection, they proved that the optimal relaxed control can be expressed in a feedback form. Furthermore, in [22] the authors used an abstract approach based on the concept of occupation measure to prove the existence of optimal relaxed controls.
1.1 The relaxed stochastic maximum principle and contributions of the paper
Optimality necessary conditions for stochastic systems in the form of Pontryagin’s maximum principle have been developed for relaxed controls in [7, 8, 28] in the case of continuous diffusions. These results have been extended to meanfield systems in [3–5]. See also [2, 12, 32] for versions of the relaxed stochastic maximum principle including doubly forwardbackward stochastic differential equations and stochastic equations driven by GBrownian motion.
Our main goal in this paper is to prove a stochastic maximum principle for relaxed controls in the case where both the drift and the diffusion coefficient are controlled. We show that the natural pathwise representation of the relaxed state process satisfies a stochastic differential equation driven by an orthogonal continuous martingale measure [15].
Note that another type of relaxation has been considered in the literature [1, 6, 35], where the authors replace the drift and the diffusion coefficients in the controlled stochastic equation by their integrals with respect to the relaxed control as in the deterministic control problems. They obtain a linear convex relaxed control problem. We prove that the main drawback of this type of relaxation is that the dynamics obtained is not continuous with respect to the control variable by providing a counterexample. As a byproduct, the relaxed and strict control problems have different value functions and the control problem obtained cannot be considered as a true relaxation. This is the first main contribution of the present paper.
Our second main result is to derive necessary conditions for optimality satisfied by an optimal relaxed control in the form of a Peng stochastic maximum principle. This is achieved through first and secondorder adjoint processes. By using the socalled Chattering lemma, the optimal relaxed control is approximated by a sequence of nearly optimal strict controls. Under pathwise uniqueness of the stochastic equation associated with the relaxed control, we prove a strong approximation result for the controlled processes. Ekeland’s variational principle then allows us to derive necessary conditions for nearoptimality satisfied by the sequence of strict controls. The final step involves proving the convergence of the corresponding adjoint processes and Hamiltonian functions, completing the proof.
Our work extends the existing maximum principles in several ways. It generalizes Peng’s principle [30] to relaxed controls and [28] to include controlled diffusion coefficient. Furthermore, assuming that a strict optimal control exists, we recover Peng’s original principle [30]. The key advantage of our result is that it applies to a natural class of controls, which is the closure of the class of strict controls, and for which the existence of an optimal solution is guaranteed. Another advantage of our method is that it is based on an approximation procedure, which could be helpful to solve numerically problems arising in practical situations. Our method relies on an approximation scheme, making it a valuable tool for addressing various numerical problems encountered in realworld control applications.
The rest of the paper is organized as follows. In the second section, we formulate the control problem and introduce the assumptions of the model. The third section is devoted to the relaxed model. In the last section, we prove rigorously the secondorder maximum principle for the relaxed control problem, representing the main contribution of this paper.
2 Formulation of the problem and notations
We consider in this paper stochastic control problems of the following type.
Let \((\Omega ,\mathcal{F},(\mathcal{F}_{t})_{t\geq 0},P)\) be a probability space equipped with a complete filtration \((\mathcal{F}_{t})_{t\geq 0}\) satisfying the usual conditions. Let \(( B_{t} ) \) be a standard ddimensional Brownian motion.
Consider a compact set \(\mathbb{A}\) in \(\mathbb{R}^{k}\), and let \(\mathcal{U}_{\mathrm{ad}}\) be the class of strict controls, which are measurable, \(\mathcal{F}_{t}\)adapted processes \(u: [ 0,T ] \times \Omega \longrightarrow \mathbb{A}\). For any u \(\in \mathcal{U}_{\mathrm{ad}}\), we consider the control problem where the controlled process is a solution of the following stochastic differential equation (SDE):
We assume that
are bounded and Borel measurable functions.
The expected cost corresponding to a strict control u is given by
where
are Borel measurable functions.
The solution X of the above SDE is called the response of the control \(u\in \mathcal{U}_{\mathrm{ad}}\). The objective of the strict control problem is to minimize the cost functional \(J(\cdot)\) over the set \(\mathcal{U}_{\mathrm{ad}}\), subject to equation (2.1). A control that solves this problem is called optimal. A strict control \(u^{\ast}\in \mathcal{U}_{\mathrm{ad}}\) is called optimal if it achieves the infimum of \(J(u)\) over \(\mathcal{U}_{\mathrm{ad}}\).
Notations
Throughout this paper, we will use the following notations.
\(x\cdot y\): the inner product of the vectors x and y.
\(\vert x \vert = \vert x_{1} \vert + \vert x_{2} \vert +\cdots+ \vert x_{n} \vert \) for a ndimensional vector \(x=(x_{1},x_{2},\ldots,x_{n})\).
\(A^{\ast}\): the transpose of a matrix A.
\(f_{x}\): the gradient of the function f with respect to x.
\(f_{xx}\): the Hessian of a scalar function f.
\(\mathcal{M}_{n\times d}(\mathbb{R} )\): the space of \(n\times d\) matrices.
\(\mathbb{A}\): a compact subset of \(\mathbb{R}^{k}\) called the action space.
\(\mathcal{U}_{\mathrm{ad}}\): the space of strict controls.
\(\mathcal{P}( [ 0,T ] \times \mathbb{A})\): the space of probability measure on the compact set \([ 0,T ] \times \mathbb{A}\).
\(\mathbb{V}\): the subset of \(\mathcal{P}( [ 0,T ] \times \mathbb{A})\) consisting of probability measures whose projection on \([ 0,T ] \) is the Lebesgue measure.
\(\mathcal{R}\): the space of relaxed controls.
\(C_{b}^{2}(\mathbb{R}^{d};\mathbb{R})\): the space of bounded continuous functions having bounded continuous first and secondorder derivatives.
\(\mathbb{D}( [ 0,T ] ,\mathbb{R}^{n})\): is the Skorokhod space of functions that are continuous from the right and have limits from the left.
Assumptions
Let us assume the following conditions on the coefficients.
\((\mathbf{A}_{1})\) The maps b, σ, h, g, and f are continuous and bounded.
\((\mathbf{A}_{2})\) b, σ, h, g admit derivatives up to the second order with respect to x, which are bounded and continuous in \(( x,a ) \).
Under the above hypothesis, (2.1) has a unique strong solution and the cost functional (2.2) is well defined from \(\mathcal{U}_{\mathrm{ad}}\) into \(\mathbb{R} \).
Note that for questions of existence of optimal controls, the probability space, Brownian motion may change with the control u. Indeed, the existence of optimal controls uses heavily the concept of weak solution of stochastic differential equations. It is worth noting that for weak solutions of the stochastic differential equations, the probability space and Brownian motion are parts of the solution. Another way to deal with weak solutions is to use martingale problems [21].
The infinitesimal generator L, associated with our controlled SDE, is the secondorder differential operator acting on functions f in \(C_{b}^{2}(\mathbb{R}^{n};\mathbb{R})\), defined by
where \(a_{ij}(t,x,u)\) denotes the generic term of the symmetric matrix \(\sigma \sigma ^{\ast}(t,x,u)\) [21].
As it is well known, weak solutions for Itô SDEs are equivalent to the existence of solutions of the corresponding martingale problems [21]. The approach by martingale problems simplifies limit analysis and avoids the relaxation complications associated with the stochastic integral part [14]. Let us define a strict control using martingale problems.
Definition 2.1
A strict control is a term \(\alpha =(\Omega ,\mathcal{F},\mathcal{F}_{t},P,u_{t},X_{t})\) such that
(1) \((\Omega ,\mathcal{F},\mathcal{F}_{t},P)\) is a probability space equipped with a filtration \((\mathcal{F}_{t})_{t\geq 0}\) satisfying the usual conditions.
(2) \(( u_{t} ) \) is an \(\mathbb{A}\)valued process, progressively measurable with respect to \((\mathcal{F}_{t})\).
(3) \((X_{t})\) is \(\mathbb{R}^{n}\)valued \(\mathcal{F}_{t}\) adapted, with continuous paths, such that
Remark 2.2
1) Condition 3) in the above definition is equivalent to saying that SDE (2.1) has a weak solution.
2) Under assumptions A\(_{\mathbf{1}}\) and A\(_{ \mathbf{2}}\) the controlled equation (2.1) has a unique strong solution for every fixed probability space and Brownian motion. So we fix the probability reference, and a strict control \(( u_{t} ) \) will be just an \(\mathbb{A}\)valued process progressively measurable with respect to \((\mathcal{F}_{t})\). There is no need to specify the probability space.
3 The relaxed control problem
3.1 A typical example
As we are going to see in a simple example, most control problems have no optimal solutions within the space of strict controls [14]. Let us consider the following wellknown example from deterministic control [11].
Minimize
over the set \(\mathbb{U}\) of measurable functions \(u:[0,1]\rightarrow \{1,1\}\), where \(X(t)\) is the solution of
We have \(\inf_{u\in \mathbb{U}}J(u)=0\).
Indeed let us consider the sequence of Rademacher functions:
It is not difficult to show that \(X^{u_{n}}(t)\leq 1/n\) and \(J(u_{n})\leq 1/n^{2}\), which implies that \(\inf_{u\in \mathbb{U}}J(u)=0\). There is, however, no control û such that \(J(\widehat{u})=0\) because this would imply that for every t, \(X^{\widehat{u}}(t)=0\); and as a consequence we obtain \(\widehat{u}_{t}=0\), which is impossible. This limit, if it exists, would be the natural candidate for optimality.
The classical way to overcome this difficulty is to introduce relaxed controls, which are measurevalued functions that describe the introduction of a stochastic parameter. Let \(dt\delta _{u(t)}(da)\) be the product measure on \([0,1]\times \{1,1\}\) such that its projection on \([0,1]\) is the Lebesgue measure and is defined as follows:
\(\delta _{u(t)}(da)\) denotes the Dirac measure concentrated at the point \(u(t)\).
The following lemma is known in deterministic control. We give its proof for the sake of completeness.
Lemma 3.1
Let \(dt\delta _{u_{{n}}(t)}(da)\) be the relaxed control associated with the Rademacher function \(u_{n}(t)\), then the sequence \(( dt\delta _{u_{{n}}(t)}(da) ) \) converges weakly to \(dt\frac{1}{2}(\delta _{1}+\delta _{1})(da)\).
Proof
It is sufficient to show that for every bounded continuous function \(f:[0,1]\times \{1,1\}\longrightarrow \mathbb{R}\)
as \(n\longrightarrow +\infty \).
Assume \(n=2m\).
\(f(t,1)\) and \(f(t,1)\) are continuous on \([0,1]\), which is bounded and closed, then they are uniformly continuous. Then, for some \(\varepsilon >0\), there exists \(N\in \mathbb{N}^{\ast}\) such that for every \(m\geq N\) such that \(\vert ts \vert <\frac{1}{m}\) we have \(\vert f(t,a)f(s,a) \vert <\varepsilon \) for \(a=1\) or \(a=1\).
This implies in particular that
and therefore
But we know that
Therefore
and
which achieves the proof.
The case where n is odd can be proved by using the same arguments. □
Remark 3.2
The sequence of Rademacher functions is a typical example of a minimizing sequence with no limit in the set of strict controls. However, its weak limit is \(dt(1/2)(\delta _{1}+\delta _{1})(da)\).
Now we can define the relaxed control as any probability measure on \([ 0,1 ] \times \{ 1,1 \} \) defined by \(\mu =dt.\mu _{t}(da)\), and the relaxed dynamics will be
The corresponding relaxed cost functional is given by
Let us point out that in the case where the relaxed control μ is associated with a strict control u, in other words \(\mu =dt.\delta _{u(t)}(da)\), then \(\mathcal{J}(\mu )=J(u)\).
It is clear that if \(\mu ^{\ast}=dt(1/2)(\delta _{1}+\delta _{1})(da)\), \(X^{\mu}(t)={\int _{0}^{t}} {\int _{ \mathbb{A}}} a.(1/2)(\delta _{1}+\delta _{1})(da)\,ds=0\), therefore \(\mathcal{J}(\mu ^{\ast })=0\). This means that \(dt(1/2)(\delta _{1}+\delta _{1})(da)\) is an optimal control in the space of relaxed controls.
3.2 The set of relaxed controls
The idea of relaxed control is to replace the \(\mathbb{A}\)valued process \(u_{t}\) with a \(\mathcal{P(}\mathbb{A})\)valued process \(\mu _{t}\), where \(\mathcal{P(}\mathbb{A})\) is the space of probability measures equipped with the topology of weak convergence. Then μ may be identified as a random product measure on \([0,T]\times \mathbb{A}\), whose projection on \([0,T]\) coincides with the Lebesgue measure.
Let \(\mathbb{V}\) be the set of product measures on \([0,T]\times \mathbb{A}\) whose projection on \([0,T]\) coincides with the Lebesgue measure dt. It is clear that every μ in \(\mathbb{V}\) may be disintegrated as \(\mu =dt.\mu _{t}(da)\), where \(\mu _{t}(da)\) is a transition probability [14].
\(\mathbb{V}\) as a closed subspace of the compact space of probability measures \(\mathcal{P}([0,T]\times \mathbb{A})\) is compact for the topology of weak convergence. In fact it can be proved that it is compact also for the topology of stable convergence, where test functions are measurable, bounded functions \(f(t,a)\) continuous in a. See [14] for further details.
Definition 3.3
A relaxed control on the filtered probability space \((\Omega ,\mathcal{F},(\mathcal{F}_{t})_{t\geq 0},P)\) is a random variable \(\mu =dt.\mu _{t}(da)\) with values in \(\mathbb{V}\) such that \(\mu _{t}(da)\) is progressively measurable with respect to \((\mathcal{F}_{t}\mathcal{)}_{t\geq 0}\) and such that for each t, \(1_{(0;t]}\) μ is \(\mathcal{F}_{t}\)measurable.
The problem now is to define rigourously the dynamics associated with a relaxed control. More precisely, since the diffusion term is controlled, one has to define the concept of martingale measure.
Let us denote by \(\mathcal{R}\) the collection of all relaxed controls.
3.3 The relaxed dynamics
When dealing with existence results, it is important to point out that the probability space and Brownian motion are parts of the relaxed control. The following definition gives a precise meaning of the notion of control.
Definition 3.4
A relaxed control is a term \(\alpha =(\Omega ,\mathcal{F},\mathcal{F}_{t},P,\mu ,X_{t})\) such that
(1) \((\Omega ,\mathcal{F},\mathcal{F}_{t},P)\) is a probability space equipped with a filtration \((\mathcal{F}_{t})_{t\geq 0}\) satisfying the
usual conditions.
(2) μ is a \(\mathbb{V}\)valued process, \(\mu (\omega ,dt,du)=dt.\mu (\omega ,t,du)\), and \(\mu (\omega ,t,du)\) is progressively measurable with respect to \((\mathcal{F}_{t})\) and such that for each t, \(1_{(0,t]}.\mu \) is \(\mathcal{F}_{t}\) adapted.
(3) \((X_{t})\) is \(\mathbb{R}^{n}\)valued \(\mathcal{F}_{t}\)adapted, with continuous paths, such that
Let us define the corresponding relaxed cost functional by
In case the relaxed control is defined by \(dt\delta _{u_{t}}(da)\), we recover the cost functional corresponding to the strict control u. More precisely, \(\mathcal{J}(dt\delta _{u_{t}}(da))=J(u)\).
It is proved in [14] that the relaxed control problem admits an optimal solution.
Theorem 3.5
Under assumption (A\(_{\mathbf{1}}\)), the relaxed optimal control problem defined by the martingale problem (3.3) and the relaxed cost functional (3.4) admits an optimal solution.
In what follows we give a pathwise representation of the solution of the relaxed martingale problem in terms of an Itô stochastic differential equation driven by an orthogonal martingale measure. Martingale measures were introduced by Walsh [34], see also [15, 25] for more details.
Definition 3.6
Let \((\Omega ,\mathcal{F},\mathcal{F}_{t},P)\) be a filtered probability space and \(M(t,B)\) be a random process, where \(B\in \mathcal{B} ( \mathbb{A} ) \) the Borel σfield of \(\mathbb{A}\). M is an \(( \mathcal{F}_{t},P)\)martingale measure if:
1)For every \(B\in \mathcal{B} ( \mathbb{A} ) , ( M(t,B) ) _{t \geq 0}\) is a square integrable martingale, \(M(0,B)=0\).
2)For every \(t>0\), \(M(t,.)\) is a σfinite \(L^{2}\)valued measure.
It is called continuous if for each \(B\in \mathcal{B} ( \mathbb{A} ) \), \(M(t,B)\) is continuous and orthogonal if \(M(t,B).M(t,C)\) is a martingale whenever \(B\cap C=\varnothing \).
Remark 3.7
When the martingale measure M is orthogonal, it is proved in [34] the existence of a random positive σfinite measure \(\mu ( dt,da ) \ \)on \([ 0,T ] \times \mathbb{A}\) such that \(\langle M(.,B),M(.,B) \rangle _{t}=\mu ( [ 0,t ] \times B ) \) for all \(t>0\), and \(B\in \mathcal{B} ( \mathbb{A} ) \). \(\mu ( dt,da ) \) is called the covariance measure of M.
Theorem 3.8
1) Let P be a solution of the martingale problem (3.3). Then P is the law of a ddimensional adapted and continuous process X defined on an extension of the space \(( \Omega ,\mathcal{F}, \mathcal{F}_{t},P ) \) and which is a solution of the following SDE starting at x:
where \(M=(M^{k})_{k=1}^{d}\) is a family of dstrongly orthogonal continuous martingale measures, each of them having intensity \(dt\mu _{t}(da)\).
2) Under assumptions (\(\mathbf{A}_{1}\)) and (\(\mathbf{A}_{2}\)), SDE (3.5) has a unique strong solution.
Proof
Let us give an outline of the proof.
1) Suppose that X is a solution of SDE (3.5) on some probability space \(( \Omega , \mathcal{F},\mathcal{F}_{t},P ) \), and let \(f\in C_{b}^{2}(\mathbb{R}^{n},\mathbb{R})\). An application of Itô’s formula gives
It is clear that \(f(X_{t})f(X_{0})\int _{0}^{t}\int _{\mathbb{A}}Lf(s,X_{s},a).\mu (s,da)\,ds=\int _{0}^{t}\int _{ \mathbb{A}}f_{x}(X_{s})\sigma (s,X_{s},a)M(ds,da)\), which is a martingale.
Conversely suppose that P is a solution of the relaxed martingale problem (3.3). This implies that
Choose \(f(x)=x_{i}\) the ith coordinate of x, where \(x=(x_{1},x_{2},\ldots,x_{n})\in B_{R}\) the ball of center 0 and radius R in \(\mathbb{R}^{d}\), \(B_{R}= \{ x\in \mathbb{R}^{d}/ \vert x \vert < R \} \). Define the first exist time of the process \(X_{t}\) from the ball \(B_{R}\), \(\tau _{R}=\inf \{ t:X_{t}\notin B_{R} \} \).
f being \(C_{b}^{2}\), it follows that \(\Gamma _{i}^{R}=X_{i}(t\wedge \tau _{R})X_{i}(0)\int _{0}^{t \wedge \tau _{R}}\int _{\mathbb{A}}b_{i}(s,X_{s},a)\mu _{s}(da)\,ds\) is a \(( P,\mathcal{F}_{t} ) \) continuous square integrable \(( P,\mathcal{F}_{t} ) \)martingale. Therefore \(\Gamma _{i}(t)=X_{i}(t)X_{i}(0)\int _{0}^{t}\int _{ \mathbb{A}}b_{i}(s,X_{s},a)\mu _{s}(da)\,ds\) is a \(( P,\mathcal{F}_{t} ) \) continuous \(( P,\mathcal{F}_{t} ) \) local martingale for any \(i=1,2,\ldots,n\).
Now, choosing \(f\in C_{b}^{2}(\mathbb{R}^{d})\) such that \(f(x)=x_{i}x_{j}\) for \(x=(x_{1},x_{2},\ldots,x_{n})\in B_{R}\), we see similarly that \(\langle \Gamma _{i},\Gamma _{j} \rangle (t)=\int _{0}^{t}\int _{\mathbb{A}}a_{ij}(s,X_{s},a)\mu _{s}(da)\,ds\) with \(( a_{ij} ) \) is the symmetric matrix \(\sigma \sigma ^{\ast}\) and \(\langle \Gamma _{i},\Gamma _{j} \rangle (t)\) is the bounded variation process such that \(\Gamma _{i}(t)\Gamma _{j}(t) \langle \Gamma _{i},\Gamma _{j} \rangle (t)\) is a \(( P,\mathcal{F}_{t} ) \) local martingale for all i, j. According to Theorem III10 in [15], on an extension of the probability space \(( \Omega ,\mathcal{F},\mathcal{F}_{t},P ) \), there exists a family of dstrongly orthogonal continuous martingale measures \(M=(M^{k})_{k=1}^{d}\), each of them having intensity \(dt\mu _{t}(da)\) such that
which achieves the proof.
2) The proof is similar to the existence and uniqueness of the solution of an SDE under Lipschitz conditions [21]. □
Remark 3.9
i) Note that the family of orthogonal martingale measures \(M=(M^{k})_{k=1}^{d}\) corresponding to the relaxed control \(\mathit{dt}\mu _{t}(da)\) is not unique.
ii) From now on, the probability space and the Brownian motion \(( B_{t} ) \) are fixed. So, a relaxed control will be defined as in Definition 3.3. The Brownian motion \(( B_{t} ) \) remains a Brownian motion on this new probability space, but the filtration is no longer the natural filtration of \(( B_{t} ) \).
Now we are able to define precisely the relaxed control problem by the following.
Minimize over \(\mathcal{R}\) the cost functional \(\mathcal{J}(\mu )\) defined by (3.4) subject to the relaxed dynamics (3.5).
3.3.1 Approximation of the relaxed control problem
In this section we will prove that the relaxed control problem is the closure of the set of strict controls. This means that if \(( u^{n} ) \) is a sequence of strict controls such that \(( \delta _{u_{t}^{n}}(da)\,dt ) \) converges to \(\mu _{t}(da)\,dt\) weakly, then the sequence of corresponding trajectories \(( X^{n} ) \) converges to \(X^{\mu}\) where \(X^{\mu}\) is the solution of relaxed SDE (3.5). This implies in particular that the map \(\mu \longrightarrow X^{\mu}\) is continuous; and as a consequence, the strict and relaxed problems have the same value function.
The following lemma [11, 29], which is classical in deterministic as well as in stochastic control, shows that the closure (for the topology of weak convergence) of the set of strict controls is exactly the set of relaxed controls. We give the proof for the sake of completeness.
Lemma 3.10
(Chattering lemma))
Let μ be a relaxed control. Then there exists a sequence of strict controls \((u^{n})\) with values in \(\mathbb{A}\) such that
Proof
For any g continuous in \([0,T]\times \mathbb{A}\), suppose that \(\mu (t,da)\) has continuous sample paths. Let \(n\geq 1\), and let \(( T_{i}= [ t_{i},s_{i} [ ) \) be subintervals of the interval \([0,T]\) of length not exceeding \(2^{n}\). Cover A by finitely many disjoint sets \(( A_{j} ) \) such that diameter \((A_{j})\leq 2^{n}\). Choose a point \((t_{i},a_{ij})\) in \(T_{i}\times A_{j}\). We have \(\sum_{j}\mu (t_{i},A_{j})=1\). Subdivide each \(T_{i}\) further into disjoint leftclosed, rightopen intervals \(T_{ij}\) such that its length is the product of \(\mu (t_{i},A_{j})\) with the length of \(T_{i}\). ∀ \(\varepsilon >0\), for n large enough, we have
Define the sequence of predictable process \(\mu ^{n}(\cdot)\) by \(\mu ^{n}(t,da)=\delta _{a_{ij}}(da)\) for \(t\in T_{ij}\). And by pathcontinuity of \(u(\cdot)\), we may increase n further if necessary to obtain
which completes the proof. In case \(\mu (t,da)\) is not continuous, we use an approximation by continuous functions. □
Proposition 3.11
1) Let \(\mu =\mu _{t}(da)\,dt\) be a relaxed control. Then there exists a continuous orthogonal martingale measure \(M(dt,da)\) whose covariance measure is given by \(\mu _{t}(da)\,dt\).
2) If we denote \(M^{n}(t,B)=\int _{0}^{t}\int _{B}\delta _{u_{s}^{n}}(da)\,dW_{s}\), where \(( u^{n} ) \) is defined as in the last lemma, then for every bounded predictable process \(\varphi :\Omega \times [ 0,T ] \times \mathbb{A} \rightarrow \mathbb{R}\), such that \(\varphi (\omega ,t,.)\) is continuous, we have
for a suitable Brownian motion B defined on an eventual extension of the probability space.
Proof
See [25] pages 196–197. □
The following theorem gives us the continuity of the controlled dynamics with respect to the control variable in the sense of law.
Theorem 3.12
Let μ be a relaxed control and \(X^{\mu}\) be the corresponding relaxed process. Assume that the relaxed SDE (3.5) has a unique weak solution. Then there exists a sequence \((u^{n})\) of strict controls such that the sequence \(( X^{u^{n}} ) \) converges in law to \(X^{\mu}\).
Proof
According to the Chattering lemma, there exists \((u^{n})\) of strict controls such that \(dt\delta _{u_{t}^{n}}(da)\) converges weakly to \(\mu _{t}(da)\,dt\), Pa.s. Let \(( X^{u^{n}} ) \) and \(X^{\mu}\) be the solutions of (3.5) corresponding to \(dt\delta _{u_{t}^{n}}(da)\) and \(dt\mu _{t}(da)\).
Since b and σ are bounded, it follows that
Therefore \(( X^{u^{n}} ) \) is tight on the space \(\mathbb{D}( [ 0,T ] ,\mathbb{R}^{d})\). Since \(( dt\delta _{u_{t}^{n}}(da) ) \) converges weakly to \(dt\mu _{s}(da)\) and \(M^{n}(dt,da)=\delta _{u_{s}^{n}}(da)\,dB_{s}\) converges to \(M(dt,da)\), and according to the uniqueness in law of the relaxed SDE (3.5), it holds that \(( X^{u^{n}} ) \) converges in law to \(X^{\mu}\). □
We will prove under pathwise uniqueness that the approximation holds in quadratic mean.
Theorem 3.13
Let μ be a relaxed control, and let X be the solution of (3.5). Assume that the coefficients of stochastic differential equation (2.1) are continuous and bounded. Assume also that pathwise uniqueness holds for (3.5). Then there exists a sequence \((u^{n})\) of strict controls such that
where \(X^{n}\) denotes the solution of the stochastic differential equation (3.5) associated with \((u^{n})\).
Proof
i) Let μ be a relaxed control, then by Lemma 3.10 there exists a sequence \((u^{n})\) such that \(\mu _{t}^{n}(da)\,dt=\delta _{u_{t}^{n}}(da)\,dt\longrightarrow \mu _{t}(da)\,dt\) in \(\mathcal{R}\), Pa.s. Let \(X^{n}\) and X be the solutions of (3.5) associated with \(\mu ^{n}\) and μ, respectively. Suppose that the result of Theorem 3.13 is false, then there exists \(\gamma >0\) such that
According to the compactness of \(\mathbb{A}\) and the boundedness of the coefficients of SDE (3.5), it follows that the family of processes
is tight. Then, by the Skorokhod selection theorem [21], there exist a probability space \((\widetilde{\Omega},\widetilde{\mathcal{F}},\widetilde{P})\) and a sequence \(\widetilde{\Gamma}^{n}=(\widetilde{\mu}^{n},\widetilde {\upsilon}^{n}, \widetilde{X}^{n},\widetilde{Y}^{n},\widetilde{M}^{n},\widetilde{N}^{n})\) defined on it such that:
i) For each \(n\in \mathbb{N} \), the laws of \(\Gamma ^{n}\) and \(\widetilde{\Gamma}^{n}\) coincide;
ii) There exists a subsequence \((\widetilde{\Gamma}^{n_{k}})\), still denoted by of \(\widetilde{\Gamma}^{n}\), which converges to Γ̃, \(\widetilde{P}a.s\)., where \(\widetilde{\Gamma}=(\widetilde{\mu},\widetilde{\upsilon},\widetilde{X},\widetilde{Y},\widetilde{M}, \widetilde {N})\). By the uniform integrability, we have
where Ẽ is the expectation with respect to P̃. According to i), we see that \(\widetilde{X}^{n}\) and \(\widetilde{Y}^{n}\) satisfy the following equations:
Since \(( \widetilde{\Gamma}^{n} ) \) converges to Γ̃, \(\widetilde{P}a.s\)., \((\widetilde{X}^{n})\) and \((\widetilde{Y}^{n})\) converge respectively to X̃ and Ỹ, which satisfy
According to Lemma 3.10, the sequence \((\mu ^{n},\mu )\) converges to \((\mu ,\mu )\) in \(\mathcal{R}^{2}\). Moreover,
Hence, \(\operatorname{law}(\widetilde{\mu},\widetilde{\upsilon})=\operatorname{law}(\mu ,\mu )\), which implies that \(\widetilde{\mu}=\widetilde{\upsilon}\), \(\widetilde{P}a.s\). By the same method, we can prove that \(\widetilde{M}=\widetilde{N}\), \(\widetilde{P}a.s\). According to the pathwise uniqueness of equation (3.5), it follows that \(\widetilde{X}=\widetilde{Y},\widetilde{P}a.s\)., which contradicts (3.7). i) is proved.
ii) This is a direct consequence of i) along with the continuity and boundedness of the functions h and g. □
Remark 3.14
1) Using the same arguments, we can replace the sequence \(( \delta _{u_{t}^{n}}(da)\,dt ) \) by any sequence \(( \mu _{t}^{n}(da)\,dt ) \) of relaxed controls converging weakly to \(\mu _{t}(da)\,dt\). This means in particular that the function \(\mu \longrightarrow X^{\mu}\) is continuous.
2) As a consequence of the last theorem, the value functions of the strict and relaxed control problems are equal. Therefore, by relaxing the control problem, the value function remains unchanged. Moreover, the relaxed control problem has an optimal solution.
3.3.2 Discussion of another relaxed model
Assume that both the drift and the diffusion coefficients are controlled. Let us consider another type of relaxation of the controlled stochastic differential equation, suggested in the literature by many authors [1, 6, 35]. Instead of relaxing the infinitesimal generator of the controlled process, the authors considered the direct relaxation of the stochastic differential equation as in deterministic control. This is carried out by integrating directly the drift and diffusion coefficient against the relaxed control, which gives the following equation:
This “relaxed” form has the advantage to be linear with respect to the control variable, with a convex compact set of controls. However, its solution has a serious drawback in that it is not continuous with respect to the control variable. As a consequence, it follows that the value functions of the strict and relaxed control problems cannot be equal. Therefore, it cannot be considered as a true relaxed model. Moreover we have no mean to prove the existence of an optimal relaxed control as the dynamics and the cost functional are not continuous with respect to the control variable.
Indeed, consider the control problem governed by the following SDE:
where admissible controls are measurable functions \(u:[0,1]\rightarrow \mathbb{A}\mathbbm{=} \{ 1,1 \} \).
The corresponding “relaxed” equation is defined by
Proposition 3.15
The solution of the controlled SDE (3.9) is not continuous in the control variable.
Proof
Consider the sequence of Rademacher functions
According to Lemma 3.1, the sequence of relaxed controls \(( dt.\delta _{u_{{n}}(t)}(da) ) \) converges weakly to \(dt.\frac{1}{2}(\delta _{1}+\delta _{1})(da)\).
Let \(X_{t}^{n}\) be the solution of SDE (3.9) associated with the relaxed control \(dt.\delta _{u_{{n}}(t)}(da)\). It is clear that
is a continuous martingale with quadratic variation \(\langle X^{n},X^{n} \rangle _{t}={\int _{0}^{t}} u_{n}^{2}(s).ds=t\). Therefore \(( X_{t}^{n} ) \) is a Brownian motion constructed possibly on an augmented probability space.
Let \(X^{\ast}\) be the relaxed state process corresponding to the limit \(\mu ^{\ast}=dt.\frac{1}{2}(\delta _{1}+\delta _{1})(da)\), then
It is obvious that the sequence of state processes \(( X_{t}^{n} ) \) does not converge in any topology to \(X_{t}^{\ast}\). Indeed
□
Remark 3.16
1) It is clear that the right limit is a Brownian motion, which could be represented as \(X^{\ast}(t)={\int _{0}^{t}} {\int _{ \mathbb{A}}} a.M(ds,da)\) where \(M ( dt,da ) ={\sum_{i=1}^{2}} \sqrt{\frac{1}{2}}\,dB_{{ s}}^{i}1_{ ( {\alpha}_{i}\in da ) }\), where \(B^{1}\) and \(B^{2}\) are independent Brownian motions constructed possibly on an augmentation of the probability space.
2) As a consequence of the last proposition, the value functions of the strict and “relaxed” control problems could be different. Moreover, even if the set \(\mathbb{V}\) is compact, there is no mean to prove the existence of an optimal control for this model.
3) Unlike the model based on SDE (3.5), the controlled stochastic equation (3.8) is driven by the martingale measure \(\mu _{t}(da)\,dB_{t}\), which is not orthogonal. Its intensity is given by \(\mu _{t}(da)\otimes \mu _{t}(da)\otimes dt\). This is a worthy martingale measure in the sense of Walsh [34].
4 Necessary conditions for optimality
We know from the previous section that an optimal relaxed control μ exists in the set \(\mathcal{R}\). This implies the existence of a filtered probability space still denoted by \((\Omega ,\mathcal{F},( \mathcal{F}_{t}) _{t\geq 0},P)\), a measurevalued control \(dt\mu _{t} ( da ) \), and an orthogonal martingale measure \(M(da,dt)\) whose covariance measure is \(dt\mu _{t} ( da ) \) such that:
and
Our goal in this section is to derive necessary conditions for optimality satisfied by the optimal relaxed control μ. According to the Chattering lemma, \(dt\mu _{t} ( da ) \) can be approximated in the sense of weak convergence by a sequence (\(u^{n}\)) of strict controls. We start by establishing the necessary conditions of near optimality that are satisfied by the strict controls \((u^{n})\). This important auxiliary result is based on Ekeland’s variational principle [13] and is interesting in itself. Indeed in most practical situations it is sufficient to characterize and compute nearly optimal controls.
Lemma 4.1
(Ekeland’s variational principle)
Let \((E,d)\) be a complete metric space and \(f:E\rightarrow \overline{\mathbb{R} }\) be lower semicontinuous and bounded from below. Given \(\varepsilon >0\), suppose that \(u^{\varepsilon}\in E\) satisfies \(f(u^{\varepsilon})\) \(\leq \inf (f)+\varepsilon \). Then, for any \(\lambda >0\), there exists \(\nu \in E\) such that

\(f(\nu )\) \(\leq f(u^{\varepsilon})\),

\(d(u^{\varepsilon},\nu )\leq \lambda \),

\(f(\nu )\leq f(\omega )+\frac{\varepsilon}{\lambda}\,d(\omega ,\nu )\) for all \(\omega \neq \nu \).
Let us endow the set \(\mathcal{U}_{\mathrm{ad}}\) of strict controls with an appropriate metric. For any u and \(v\in \mathcal{U}_{\mathrm{ad}}\), we set
where \(P\otimes dt\) is the product measure of P with the Lebesgue measure dt.
Remark 4.2
It is well known that \((\mathcal{U}_{\mathrm{ad}},d)\) is a complete metric space and that the cost functional J is continuous from \(\mathcal{U}_{\mathrm{ad}}\) into \(\mathbb{R} \), see [26].
Now, let \(\mu \in \mathcal{R}\) be an optimal relaxed control and denote by X the solution of (4.1) controlled by μ. From Lemma 3.10 and Theorem 3.13, there exists a sequence \((u^{n})\) of strict controls such that
and
where \(X^{n}\) is the solution of (3.5) corresponding to \(\mu ^{n}=\delta _{u_{t}^{n}}(da)\,dt\).
Let us introduce the usual Hamiltonian of the system
where \(A^{\ast}\) denotes the transpose of the vector or matrix A.
We define as in [36] by \(( p,q ) \) and \(( P,Q ) \) the first and secondorder adjoint processes satisfying the following backward SDEs, assuming that \(( \mathcal{F}_{t} ) \) is the natural filtration of the Brownian motion:
where \(b_{x}(t)=b_{x}(t,X_{t},u_{t})\) and \(\sigma _{x}^{j}(t)=\sigma _{x}^{j}(t,X_{t},u_{t})\) and \(h_{x}(t)=h_{x}(t,X_{t},u_{t})\).
Under conditions \((\mathbf{A}_{1})\) and \((\mathbf{A}_{2})\), BSDEs (4.3) and (4.4) have unique solutions satisfying the following estimates:
Remark 4.3
In case \(( \mathcal{F}_{t} ) \) is not necessarily the natural filtration of the Brownian motion, we must add in the backward equations (4.3) and (4.4) two cadlag martingales that are orthogonal to the Brownian motion. This comes from the Itô representation theorem for Brownian martingales.
4.1 Necessary conditions for near optimality
The generalized Hamiltonian \(\mathcal{H}\) associated with a strict control u and the corresponding state process X is defined as in [36] by
where \(( p,q ) \) and \(( P,Q ) \) are solutions of the adjoint equations (4.3) and (4.4). The following theorem gives necessary conditions for near optimality for the strict control \(u^{n}\) in terms of an approximate maximum principle. See [38] Theorem 4.1 for a complete proof of this intermediary result.
Proposition 4.4
There exists a sequence of strict controls \((u^{n})\) such that
and there exist unique adapted solutions \(( p^{n},q^{n} ) \) and \(( P^{n},Q^{n} ) \) of the adjoint equations (4.3) and (4.4), corresponding to the admissible pair \((u^{n},X^{n})\), such that for any \(\gamma \in [ 0,1/3 ) \)
Proof
Let us give the outline of the proof. According to the optimality of μ and the Chattering lemma, there exist a sequence \((\varepsilon _{n})\) of positive numbers with \(\lim_{n\rightarrow \infty}\varepsilon _{n}=0\) and a sequence of strict controls \((u^{n})\) such that \((u^{n})\)
According to a suitable version of Lemma 4.1 with \(\lambda =\varepsilon ^{\frac{2}{3}}\),
Let us define the perturbation
From (4.6) we have
Using the definition of d, it holds that
Let us denote by \(x^{n,h}\) the solution of (2.1) corresponding to \(u^{n,h}\), which is defined in (4.7). To get the desired variational inequality we differentiate the function \(J(u^{n,h})\) with respect to h at \(h=0\). See [38] Theorem 4.1 for details. □
4.2 The relaxed maximum principle
Let X be the corresponding optimal state process associated with the optimal relaxed control μ, and \(( p,q ) \) and \(( P,Q ) \) be the solutions of the adjoint equations (4.9) and (4.10) associated with \(( \mu ,X ) \). We assume that \(( \mathcal{F}_{t} ) \) is the natural filtration of the Brownian motion.
and
where \(\overline{k}=k(t,X_{t},\mu _{t})={\int _{A}} k(t,X_{t},a)\mu _{t}(da)\), and k stands to be \(b_{x}\), \(\sigma _{x}\), \(f_{x}\), \(h_{x}\), and \(H_{xx}\).
The generalized Hamiltonian function associated with the optimal pair \(( \mu ,X ) \) is defined by
Theorem 4.5
(Relaxed maximum principle)
Assume (\(\mathbf{A}_{1}\)) and (A2). Let \(( \mu ,X ) \) be an optimal pair, then there exist unique adapted solutions \((p,q)\) and \((P,Q)\) of the adjoint equations (4.9) and (4.10), respectively, such that
The proof of this theorem is based on the following stability theorem of adjoint processes with respect to the control variable.
Theorem 4.6
(Stability theorem for BSDEs)
Let \((p^{n},q^{n})\), \((P^{n},Q^{n})\), and \(( \textit{resp}.(p,q),(P,Q) ) \) be the solutions of (4.3) and (4.4) associated with the pair \(( u^{n},X^{n} ) \) (resp the solutions of (4.9) and (4.10) associated with the pair \(( \mu ,X ) \). Then we have
and
Proof
i) Let us write down the drivers of the firstorder adjoint equations (4.3) and (4.9) corresponding to \((u^{n},X^{n})\) and \(( \mu ,X)\).
where
By using the result of Hu and Peng [20], Theorem 2.1, it is sufficient to show that
Indeed, we have
Let us deal with the first term on the righthand side of (4.14).
\(b_{x}\) being Lipschitz in x and \(( X_{t}^{n} ) \) converges to \(X_{t}\) uniformly in t in probability imply that the first term on the righthand side of (4.15) converges in probability to 0.
In addition, we have \(E ( \sup_{0\leq t\leq T} \vert p(t) \vert ^{2} ) <+\infty \), therefore \(\sup_{0\leq t\leq T} \vert p(t) \vert <+\infty \), Pa.s, which implies the existence of a Pnegligible set N such that for each \(\omega \notin N\) there exist \(M(\omega )<+\infty \) s.t. \(\sup_{0\leq t\leq T} \vert p(t) \vert \leq M( \omega )\).
In particular, for each \(\omega \notin N\), the function \(b_{x}(t,X_{t},E(X_{t}),a)p(t).1_{ [ 0,t ] }\) is a measurable bounded function in \((t,a)\) and continuous in a; therefore it is a test function for the stable convergence. Hence, by using the fact that \(( \delta _{u_{t}^{n}}(da)\,dt ) \) converges in \(\mathbb{V}\) to \(\mu _{t}(da)\,dt\), Pa.s., it follows that the second term on the righthand side tends to 0, Pa.s.
The other terms containing \(p(t)\) can be handled by using the same techniques.
The terms in (4.14) containing \(q(t)\) can be treated similarly. However, one should pay a little more attention as \(q(t)\) is just square integrable (in \((t,\omega )\)). More precisely,
The first integral on the righthand side may be treated by using similar arguments as previously as the function \(( \sigma _{x}^{n}(t)\overline {\sigma}_{x}(t) ) q(t)1_{ \{ \vert q(t) \vert \leq N \} }\) is measurable bounded and continuous in a. The second term tends to 0 by Chebyshev’s inequality using the square integrability of \(q(t)\).
ii) is proved by using similar arguments. □
Proof
of Theorem 4.5. The main result is proved by passing to the limit in inequality (4.5) and using Theorem 4.6 to get the desired inequality (4.11). □
Corollary 4.7
Under the same conditions as in Theorem 4.5it holds that
where \(\mathcal{H} ( t,X_{t},\upsilon ) =\int _{\mathbb{A}}\mathcal{H} ( t,X_{t},a ) \upsilon (da)\) and \(\mathcal{P} ( \mathbb{A} ) \) is the space of probability measures on \(\mathbb{A}\).
Proof
Since \(\{ \delta _{a}(da);\text{ }a\in \mathbb{A} \} \subset \) \(\mathbb{P} ( \mathbb{A} ) \), it is clear that the inequality
is obvious. Let us prove the inequality from the other sense. If \(\upsilon \in \mathbb{P} ( A ) \) is a probability measure on \(\mathbb{A}\), then
where \(\operatorname{conv}(B)\) is the convex hull of B.
Hence, by using Fubini’s theorem, it holds that
which implies that
□
Remark
Since \(\mathcal{P}(\mathbb{A})\) is a subspace of \(\mathbb{V}\) whose elements are constant (in \(( \omega ,t ) \)) relaxed controls, then (4.16) may be replaced by
Corollary 4.8
(Pontryagin’s relaxed maximum principle). Under the same conditions as in Theorem 4.5, there exists a Lebesgue negligible subset N in the interval \([ 0,T ] \) such that for any t not in N it holds that
Proof
Let \(\varepsilon \in ] 0,T [ \) and \(B\in \mathcal{F}_{\varepsilon}\), for small \(h>0\) define the relaxed control
where υ is a probability measure on \(\mathbb{A}\). It follows from (4.16) that
Therefore passing at the limit as h tends to zero, we obtain
for any ε not in some Lebesgue null set N.
The last inequality is true for all \(B\in \mathcal{F}_{\varepsilon ,}\), then for any bounded \(\mathcal{F}_{\varepsilon}\)measurable random variable F it holds that
which leads to
We conclude by using the measurability of the Hamiltonian with respect to \(\mathcal{F}_{\varepsilon}\). □
4.3 Example
To illustrate our results, we present an example inspired from [36]. To simplify the notations, we suppose that the problem is one dimensional. Assume that the dynamics is given by
and the cost functional is defined by
The strict controls are measurable functions from \([ 0,1 ] \) to the set \(\{ 1,1 \} \).
By replacing \(x_{t}={\int _{0}^{1}} u(s)\,dB_{{s}}\) in the cost functional, we obtain
Since \(( \frac{3}{2}t ) >0\) for any \(t\in [ 0,1 ] \), it is clear that \(J(u)\) attains its minimum for \(\overline{u}(t)=0\), with the state process \(\overline{x}(t)=0\). But this is impossible as the strict controls take only the values −1 and 1.
Let us define the relaxed optimal control problem. As the action space \(\mathbb{A}= \{ 1,1 \} \), a relaxed control is defined explicitly by
where \(\alpha (t)\) is a measurable function such that \(0\leq \alpha (t)\leq 1\).
The cost functional associated with a relaxed control is then defined by
The cost functional attains its minimum at \(\alpha (t)=\frac{1}{2}\) and the optimal control is given by
Let us verify that this optimal control satisfies the necessary conditions of Theorem 4.5.
The first and secondorder adjoint processes \((p_{t},q_{t})\) and \((P_{t},Q_{t})\) are the unique adapted solutions of first and secondorder adjoint equations. The unique solutions are \((p_{t},q_{t})=(0,0)\) and \((P_{t},Q_{t})=(2t4,0)\).
It follows that the generalized Hamiltonian is given by
Therefore the generalized Hamiltonian for relaxed controls is defined by
\((2t3)\) being negative for \(t\in [ 0,1 ] \), it follows that the generalized Hamiltonian is concave, then attains it maximum at \(\alpha (t)=\frac{1}{2}\).
Therefore the relaxed optimal control \(dt\mu (da)=dt ( \frac{1}{2}\delta _{1}(da)+\frac{1}{2}\delta _{1}(da) ) \) satisfies the maximum principle.
Data availability
Not applicable.
References
Ahmed, N.U., Charalambous, C.D.: Stochastic minimum principle for partially observed systems subject to continuous and jump diffusion processes and driven by relaxed controls. SIAM J. Control Optim. 51(4), 3235–3257 (2013)
AlHussein, A., Gherbal, B.: Necessary and sufficient optimality conditions for relaxed and strict control of forwardbackward doubly SDEs with jumps under full and partial information. J. Syst. Sci. Complex. 33(6), 1804–1846 (2020)
Bahlali, K., Mezerdi, M., Mezerdi, B.: Existence of optimal controls for systems governed by meanfield stochastic differential equations. Afr. Stat. 9(1), 627–645 (2014)
Bahlali, K., Mezerdi, M., Mezerdi, B.: Existence and optimality conditions for relaxed meanfield stochastic control problems. Syst. Control Lett. 102, 1–8 (2017)
Bahlali, K., Mezerdi, M., Mezerdi, B.: On the relaxed meanfield stochastic control problem. Stoch. Dyn. 18(3), 1850024 (2018)
Bahlali, S.: Necessary and sufficient optimality conditions for relaxed and strict control problems. SIAM J. Control Optim. 47(4), 2078–2095 (2008)
Bahlali, S., Djehiche, B., Mezerdi, B.: The relaxed stochastic maximum principle in singular optimal control of diffusions. SIAM J. Control Optim. 46(2), 427–444 (2007)
Bahlali, S., Djehiche, B., Mezerdi, B.: Approximation and optimality necessary conditions in relaxed stochastic control problems. J. Appl. Math. Stoch. Anal. 2006, 72762 (2006)
Becker, H., Mandrekar, V.: On the existence of optimal random controls. J. Math. Mech. 18, 1151–1166 (1969)
Bensoussan, A.: Lectures on stochastic control. Nonlinear filtering and stochastic control. In: Lecture Notes in Math., Cortona, 1981, vol. 972, pp. 1–62. Springer, Berlin (1982)
Borkar, V.S.: Optimal Control of Diffusion Processes. Pitman Research Notes in Math. Series, vol. 203. Longman, Harlow (1989)
Dou, C., Wei, L., Liu, X.: Stochastic maximum principle for delayed backward doubly relaxed stochastic control problem and applications. In: 2020 IEEE 3rd International Conference of Safe Production and Informatization (IICSPI), Chongqing City, China, 2020, pp. 343–351. https://doi.org/10.1109/IICSPI51290.2020.9332341
Ekeland, I.: Nonconvex minimization problems. Bull. Am. Math. Soc. 1(3), 443–474 (1979)
El Karoui, N., Du Huu, N., JeanblancPicqué, M.: Compactification methods in the control of degenerate diffusions: existence of an optimal control. Stochastics 20(3), 169–219 (1987)
El Karoui, N., Méléard, S.: Martingale measures and stochastic calculus. Probab. Theory Relat. Fields 84(1), 83–101 (1990)
Fleming, W.H.: Generalized solutions in optimal stochastic control. In: Differential Games and Control Theory II, Proceedings of 2nd Conference, Univ. of Rhode Island, Kingston, RI, 1976. Lect. Notes in Pure and Appl. Math., vol. 30, pp. 147–165. Dekker, New York (1977)
Haussmann, U.G.: General necessary conditions for optimal control of stochastic systems. Math. Program. Stud. 6, 30–48 (1976)
Haussmann, U.G.: Existence of optimal Markovian controls for degenerate diffusions. In: Stochastic Differential Systems, Bad Honnef, 1985. Lect. Notes Control Inf. Sci., vol. 78, pp. 171–186. Springer, Berlin (1986)
Haussmann, U.G., Lepeltier, J.P.: On the existence of optimal controls. SIAM J. Control Optim. 28(4), 851–902 (1990)
Hu, Y., Peng, S.: A stability theorem of backward stochastic differential equations and its application. C. R. Acad. Sci., Ser. 1 Math. 324(9), 1059–1064 (1997)
Ikeda, N., Watanabe, S.: Stochastic Differential Equations and Diffusion Processes, 2nd edn. NorthHolland Mathematical Library, vol. 24. NorthHolland, Amsterdam (1989)
Kurtz, T.G., Stockbridge, R.H.: Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim. 36(2), 609–653 (1998)
Kushner, H.J.: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control Optim. 10, 550–565 (1972)
Kushner, H.J.: Existence results for optimal stochastic controls. Existence theory in the calculus of variations and optimal control. J. Optim. Theory Appl. 15, 347–359 (1975)
Méléard, S.: Representation and approximation of martingale measures. In: Stochastic Partial Differential Equations and Their Applications, pp. 188–199. Springer, Berlin (1992)
Mezerdi, B.: Necessary conditions for optimality for a diffusion with a nonsmooth drift. Stochastics 24(4), 305–326 (1988)
Mezerdi, B., Bahlali, S.: Approximation in optimal control of diffusion processes. Random Oper. Stoch. Equ. 8(4), 365–372 (2000)
Mezerdi, B., Bahlali, S.: Necessary conditions for optimality in relaxed stochastic control problems. Stoch. Stoch. Rep. 73(3–4), 201–218 (2002)
Mezerdi, M.A.: Compactification in optimal control of McKeanVlasov stochastic differential equations. Optim. Control Appl. Methods 42(4), 1161–1177 (2021)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28(4), 966–979 (1990)
Pham, H.: Continuoustime stochastic control and optimization with financial applications. In: Series Stochastic Modeling and Applied Probability, vol. 61. Springer, Berlin (2009)
Redjil, A., Gherbal, H.B., Kebiri, O.: Existence of relaxed stochastic optimal control for $G$SDEs with controlled jumps. Stoch. Anal. Appl. 41(1), 115–133 (2023)
Tang, S., Li, X.: Necessary conditions for optimal control of stochastic systems with random jumps. SIAM J. Control Optim. 32(5), 1447–1475 (1994)
Walsh, J.B.: An introduction to stochastic partial differential equations. In: École d’été de probabilités de SaintFlour, XIV–1984. Lecture Notes in Math., vol. 1180, pp. 265–439. Springer, Berlin (1986)
Yinggu, C., Tianyang, N., Zhen, W.: The stochastic maximum principle for relaxed control problem with regimeswitching. Syst. Control Lett. 169, 105391 (2022)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43. Springer, Berlin (1999)
Young, L.C.: Lectures on the Calculus of Variations and Optimal Control Theory, vol. 304. Am. Math. Soc., Providence (1980)
Zhou, X.Y.: Stochastic nearoptimal controls: necessary and sufficient conditions for nearoptimality. SIAM J. Control Optim. 36(3), 929–947 (1998)
Acknowledgements
The authors would like to thank the anonymous referees for deep and careful reading of the manuscript, which led to a substantial improvement of the paper.
Funding
The second author (Brahim Mezerdi) would like to acknowledge the support provided by the Deanship of Scientific Research (DSR) at King Fahd University of Petroleum and Minerals (KFUPM), KSA, for funding this work through Project No: SB201017.
Author information
Authors and Affiliations
Contributions
Both authors have contributed in the subject, analysis, proofs of the main results and in writing the paper. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mezerdi, M., Mezerdi, B. On the maximum principle for relaxed control problems of nonlinear stochastic systems. Adv Cont Discr Mod 2024, 8 (2024). https://doi.org/10.1186/s1366202403803w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s1366202403803w