On the maximum principle for relaxed control problems of nonlinear stochastic systems

We consider optimal control problems for a system governed by a stochastic diﬀerential equation driven by a d-dimensional Brownian motion where both the drift and the diﬀusion coeﬃcient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this diﬃculty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic diﬀerential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng’s maximum principle to relaxed control problems. We show that relaxing the drift and diﬀusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable.


Introduction
Our main goal in this paper is to prove a stochastic maximum principle for relaxed controls in the case where both the drift and the diffusion coefficient are controlled.
It is well known that the two main approaches to handling optimal control problems are the dynamic programming by Bellman and the maximum principle by Pontryagin [36].
The maximum principle provides a set of necessary conditions for optimality that an optimal control must satisfy, as detailed in [36].These conditions include a forward equation for the state process, a backward equation for the adjoint variable, and minimization of the Hamiltonian function.
Within stochastic control, two main approaches for a stochastic maximum principle emerge, based on the solution concept (weak/strong) and control type (open-loop/feed-back).The first approach, for strong solutions with open-loop controls, was established by [23] using spike variations.In [17] the author employed martingale methods and the Girsanov theorem to derive a maximum principle for weak solutions with feedback controls.
In [10] the author addressed systems where the diffusion coefficient depends on the control variable, utilizing convex perturbations and the first-order adjoint variable.His result constitutes a weak maximum principle with the variational inequality applied to the Gâteaux derivative of the Hamiltonian.In [30], a global maximum principle was established for a nonconvex domain and controlled diffusion coefficient, involving the introduction of a second-order adjoint process.This extension was further developed for jump-diffusion processes in [33].
Pontryagin's maximum principle has proven widespread application in solving problems related to mathematical finance and portfolio optimization, as shown in [31].We recommend [36] for a comprehensive overview and detailed references on the subject.
The main motivation behind relaxed controls lies in their property to guarantee the existence of optimal solutions in this class.This concept originated with Young's work [37] on generalized solutions in the calculus of variations, leading to the notion of Young measure.Subsequently, this framework was extended to deterministic control theory, giving rise to the concept of relaxed control.A key challenge in nonconvex control problems arises from the lack of closure of the set of strict controls under simple convergence of measurable functions.Relaxed controls elegantly address this challenge by replacing strict controls with random Dirac probability measures.This effectively transforms the set of strict controls into a compact subset of probability measures, ensuring closure under the topology of weak convergence.This "relaxation" of the convergence requirement enables us to formulate the optimal control problem as a continuous function optimization over a compact metric space, guaranteeing the existence of an optimal solution.The authors in [9,16,24] established the first existence results of relaxed controls for stochastic differential equations with uncontrolled diffusion coefficients.Subsequently, more complex systems with controlled diffusion coefficients were tackled by [14,18,19].Using Krylov's method of Markovian selection, they proved that the optimal relaxed control can be expressed in a feedback form.Furthermore, in [22] the authors used an abstract approach based on the concept of occupation measure to prove the existence of optimal relaxed controls.

The relaxed stochastic maximum principle and contributions of the paper
Optimality necessary conditions for stochastic systems in the form of Pontryagin's maximum principle have been developed for relaxed controls in [7,8,28] in the case of continuous diffusions.These results have been extended to mean-field systems in [3][4][5].See also [2,12,32] for versions of the relaxed stochastic maximum principle including doubly forward-backward stochastic differential equations and stochastic equations driven by G-Brownian motion.
Our main goal in this paper is to prove a stochastic maximum principle for relaxed controls in the case where both the drift and the diffusion coefficient are controlled.We show that the natural pathwise representation of the relaxed state process satisfies a stochastic differential equation driven by an orthogonal continuous martingale measure [15].
Note that another type of relaxation has been considered in the literature [1,6,35], where the authors replace the drift and the diffusion coefficients in the controlled stochastic equation by their integrals with respect to the relaxed control as in the deterministic control problems.They obtain a linear convex relaxed control problem.We prove that the main drawback of this type of relaxation is that the dynamics obtained is not continuous with respect to the control variable by providing a counterexample.As a byproduct, the relaxed and strict control problems have different value functions and the control problem obtained cannot be considered as a true relaxation.This is the first main contribution of the present paper.
Our second main result is to derive necessary conditions for optimality satisfied by an optimal relaxed control in the form of a Peng stochastic maximum principle.This is achieved through first-and second-order adjoint processes.By using the so-called Chattering lemma, the optimal relaxed control is approximated by a sequence of nearly optimal strict controls.Under pathwise uniqueness of the stochastic equation associated with the relaxed control, we prove a strong approximation result for the controlled processes.Ekeland's variational principle then allows us to derive necessary conditions for near-optimality satisfied by the sequence of strict controls.The final step involves proving the convergence of the corresponding adjoint processes and Hamiltonian functions, completing the proof.
Our work extends the existing maximum principles in several ways.It generalizes Peng's principle [30] to relaxed controls and [28] to include controlled diffusion coefficient.Furthermore, assuming that a strict optimal control exists, we recover Peng's original principle [30].The key advantage of our result is that it applies to a natural class of controls, which is the closure of the class of strict controls, and for which the existence of an optimal solution is guaranteed.Another advantage of our method is that it is based on an approximation procedure, which could be helpful to solve numerically problems arising in practical situations.Our method relies on an approximation scheme, making it a valuable tool for addressing various numerical problems encountered in real-world control applications.
The rest of the paper is organized as follows.In the second section, we formulate the control problem and introduce the assumptions of the model.The third section is devoted to the relaxed model.In the last section, we prove rigorously the second-order maximum principle for the relaxed control problem, representing the main contribution of this paper.

Formulation of the problem and notations
We consider in this paper stochastic control problems of the following type.
Let ( , F, (F t ) t≥0 , P) be a probability space equipped with a complete filtration (F t ) t≥0 satisfying the usual conditions.Let (B t ) be a standard d-dimensional Brownian motion.
Consider a compact set A in R k , and let U ad be the class of strict controls, which are measurable, F t -adapted processes u : [0, T] × − → A. For any u ∈ U ad , we consider the control problem where the controlled process is a solution of the following stochastic differential equation (SDE): are bounded and Borel measurable functions.
The expected cost corresponding to a strict control u is given by where are Borel measurable functions.The solution X of the above SDE is called the response of the control u ∈ U ad .The objective of the strict control problem is to minimize the cost functional J(•) over the set U ad , subject to equation (2.1).A control that solves this problem is called optimal.A strict control u * ∈ U ad is called optimal if it achieves the infimum of J(u) over U ad .
Notations Throughout this paper, we will use the following notations.
x • y: the inner product of the vectors x and y.
A * : the transpose of a matrix A. f x : the gradient of the function f with respect to x. f xx : the Hessian of a scalar function f .M n×d (R): the space of n × d matrices.
A: a compact subset of R k called the action space.U ad : the space of strict controls.P([0, T] × A): the space of probability measure on the compact set [0, T] × A. V: the subset of P([0, T] × A) consisting of probability measures whose projection on [0, T] is the Lebesgue measure.
R: the space of relaxed controls.C 2 b (R d ; R): the space of bounded continuous functions having bounded continuous firstand second-order derivatives.
D([0, T], R n ): is the Skorokhod space of functions that are continuous from the right and have limits from the left.
Assumptions Let us assume the following conditions on the coefficients.
(A 1 ) The maps b, σ , h, g, and f are continuous and bounded.(A 2 ) b, σ , h, g admit derivatives up to the second order with respect to x, which are bounded and continuous in (x, a).
Under the above hypothesis, (2.1) has a unique strong solution and the cost functional (2.2) is well defined from U ad into R.
Note that for questions of existence of optimal controls, the probability space, Brownian motion may change with the control u.Indeed, the existence of optimal controls uses heavily the concept of weak solution of stochastic differential equations.It is worth noting that for weak solutions of the stochastic differential equations, the probability space and Brownian motion are parts of the solution.Another way to deal with weak solutions is to use martingale problems [21].
The infinitesimal generator L, associated with our controlled SDE, is the second-order differential operator acting on functions f in C 2 b (R n ; R), defined by where a ij (t, x, u) denotes the generic term of the symmetric matrix σ σ * (t, x, u) [21].
As it is well known, weak solutions for Itô SDEs are equivalent to the existence of solutions of the corresponding martingale problems [21].The approach by martingale problems simplifies limit analysis and avoids the relaxation complications associated with the stochastic integral part [14].Let us define a strict control using martingale problems.Definition 2.1 A strict control is a term α = ( , F, F t , P, u t , X t ) such that (1) ( , F, F t , P) is a probability space equipped with a filtration (F t ) t≥0 satisfying the usual conditions.
(2) (u t ) is an A-valued process, progressively measurable with respect to (F t ). ( Remark 2.2 1) Condition 3) in the above definition is equivalent to saying that SDE (2.1) has a weak solution.
2) Under assumptions A 1 and A 2 the controlled equation (2.1) has a unique strong solution for every fixed probability space and Brownian motion.So we fix the probability reference, and a strict control (u t ) will be just an A-valued process progressively measurable with respect to (F t ).There is no need to specify the probability space.

The relaxed control problem 3.1 A typical example
As we are going to see in a simple example, most control problems have no optimal solutions within the space of strict controls [14].Let us consider the following well-known example from deterministic control [11]. Minimize We have inf u∈U J(u) = 0.
Indeed let us consider the sequence of Rademacher functions: , which implies that inf u∈U J(u) = 0.There is, however, no control u such that J( u) = 0 because this would imply that for every t, X u (t) = 0; and as a consequence we obtain u t = 0, which is impossible.This limit, if it exists, would be the natural candidate for optimality.
The classical way to overcome this difficulty is to introduce relaxed controls, which are measure-valued functions that describe the introduction of a stochastic parameter.Let dtδ u(t) (da) be the product measure on [0, 1] × {-1, 1} such that its projection on [0, 1] is the Lebesgue measure and is defined as follows: δ u(t) (da) denotes the Dirac measure concentrated at the point u(t).The following lemma is known in deterministic control.We give its proof for the sake of completeness.

Lemma 3.1 Let dtδ u n (t) (da) be the relaxed control associated with the Rademacher function u n (t), then the sequence
Proof It is sufficient to show that for every bounded continuous function f f (t, -1) and f (t, 1) are continuous on [0, 1], which is bounded and closed, then they are uniformly continuous.Then, for some ε > 0, there exists and therefore But we know that Therefore and which achieves the proof.
The case where n is odd can be proved by using the same arguments.
Remark 3.2 The sequence of Rademacher functions is a typical example of a minimizing sequence with no limit in the set of strict controls.However, its weak limit is dt(1/2)(δ -1 + δ 1 )(da).
Now we can define the relaxed control as any probability measure on [0, 1] × {-1, 1} defined by μ = dt.μt (da), and the relaxed dynamics will be The corresponding relaxed cost functional is given by Let us point out that in the case where the relaxed control μ is associated with a strict control u, in other words μ = dt.δu(t) (da), then (da) ds = 0, therefore J (μ * ) = 0.This means that dt(1/2)(δ -1 + δ 1 )(da) is an optimal control in the space of relaxed controls.

The set of relaxed controls
The idea of relaxed control is to replace the A-valued process u t with a P(A)-valued process μ t , where P(A) is the space of probability measures equipped with the topology of weak convergence.Then μ may be identified as a random product measure on [0, T] × A, whose projection on [0, T] coincides with the Lebesgue measure.
Let V be the set of product measures on [0, T] × A whose projection on [0, T] coincides with the Lebesgue measure dt.It is clear that every μ in V may be disintegrated as μ = dt.μt (da), where μ t (da) is a transition probability [14].
V as a closed subspace of the compact space of probability measures P([0, T] × A) is compact for the topology of weak convergence.In fact it can be proved that it is compact also for the topology of stable convergence, where test functions are measurable, bounded functions f (t, a) continuous in a. See [14] for further details.Definition 3.3 A relaxed control on the filtered probability space ( , F, (F t ) t≥0 , P) is a random variable μ = dt.μt (da) with values in V such that μ t (da) is progressively measurable with respect to (F t ) t≥0 and such that for each t, 1 (0;t] μ is F t -measurable.
The problem now is to define rigourously the dynamics associated with a relaxed control.More precisely, since the diffusion term is controlled, one has to define the concept of martingale measure.
Let us denote by R the collection of all relaxed controls.

The relaxed dynamics
When dealing with existence results, it is important to point out that the probability space and Brownian motion are parts of the relaxed control.The following definition gives a precise meaning of the notion of control.

Definition 3.4
A relaxed control is a term α = ( , F, F t , P, μ, X t ) such that (1) ( , F, F t , P) is a probability space equipped with a filtration (F t ) t≥0 satisfying the usual conditions.
(2) μ is a V-valued process, μ(ω, dt, du) = dt.μ(ω,t, du), and μ(ω, t, du) is progressively measurable with respect to (F t ) and such that for each t, 1 (0,t] .μ is F t -adapted. ( Let us define the corresponding relaxed cost functional by In case the relaxed control is defined by dtδ u t (da), we recover the cost functional corresponding to the strict control u.More precisely, J (dtδ u t (da)) = J(u).
It is proved in [14] that the relaxed control problem admits an optimal solution.In what follows we give a pathwise representation of the solution of the relaxed martingale problem in terms of an Itô stochastic differential equation driven by an orthogonal martingale measure.Martingale measures were introduced by Walsh [34], see also [15,25] for more details.Definition 3.6 Let ( , F, F t , P) be a filtered probability space and M(t, B) be a random process, where B ∈ B(A) the Borel σ -field of A. M is an (F t , P)-martingale measure if: 1)For every B ∈ B(A), (M(t, B)) t≥0 is a square integrable martingale, M(0, B) = 0.
2)For every Remark 3.7 When the martingale measure M is orthogonal, it is proved in [34] the existence of a random positive σ -finite measure μ(dt, da) on [0,T] × A such that M(., B), M(., B) t = μ([0, t] × B) for all t > 0, and B ∈ B(A).μ(dt, da) is called the covariance measure of M. Theorem 3.8 1) Let P be a solution of the martingale problem (3.3).Then P is the law of a d-dimensional adapted and continuous process X defined on an extension of the space ( , F, F t , P) and which is a solution of the following SDE starting at x: k=1 is a family of d-strongly orthogonal continuous martingale measures, each of them having intensity dtμ t (da).
Proof Let us give an outline of the proof.1) Suppose that X is a solution of SDE ( 3.5) on some probability space ( , F, F t , P), and ).An application of Itô's formula gives Conversely suppose that P is a solution of the relaxed martingale problem (3.3).This implies that A b i (s, X s , a)μ s (da) ds is a (P, F t ) continuous square integrable (P, F t )-martingale.Therefore i (t) = X i (t) -X i (0)t 0 A b i (s, X s , a)μ s (da) ds is a (P, F t ) continuous (P, F t )-local martingale for any i = 1, 2, . . ., n.
ds with (a ij ) is the symmetric matrix σ σ * and i , j (t) is the bounded variation process such that i (t) j (t)i , j (t) is a (P, F t )local martingale for all i, j.According to Theorem III-10 in [15], on an extension of the probability space ( , F, F t , P), there exists a family of d-strongly orthogonal continuous martingale measures M = (M k ) d k=1 , each of them having intensity dtμ t (da) such that which achieves the proof.
2) The proof is similar to the existence and uniqueness of the solution of an SDE under Lipschitz conditions [21].Remark 3.9 i) Note that the family of orthogonal martingale measures M = (M k ) d k=1 corresponding to the relaxed control dtμ t (da) is not unique.
ii) From now on, the probability space and the Brownian motion (B t ) are fixed.So, a relaxed control will be defined as in Definition 3.3.The Brownian motion (B t ) remains a Brownian motion on this new probability space, but the filtration is no longer the natural filtration of (B t ).Now we are able to define precisely the relaxed control problem by the following.Minimize over R the cost functional J (μ) defined by (3.4) subject to the relaxed dynamics (3.5).

Approximation of the relaxed control problem
In this section we will prove that the relaxed control problem is the closure of the set of strict controls.This means that if (u n ) is a sequence of strict controls such that (δ u n t (da) dt) converges to μ t (da) dt weakly, then the sequence of corresponding trajectories (X n ) converges to X μ where X μ is the solution of relaxed SDE (3.5).This implies in particular that the map μ − → X μ is continuous; and as a consequence, the strict and relaxed problems have the same value function.
The following lemma [11,29], which is classical in deterministic as well as in stochastic control, shows that the closure (for the topology of weak convergence) of the set of strict controls is exactly the set of relaxed controls.We give the proof for the sake of completeness.
Lemma 3.10 (Chattering lemma)) Let μ be a relaxed control.Then there exists a sequence of strict controls (u n ) with values in A such that which completes the proof.In case μ(t, da) is not continuous, we use an approximation by continuous functions.
Proposition 3.11 1) Let μ = μ t (da) dt be a relaxed control.Then there exists a continuous orthogonal martingale measure M(dt, da) whose covariance measure is given by μ t (da) dt.
2) If we denote M n (t, B) = t 0 B δ u n s (da) dW s , where (u n ) is defined as in the last lemma, then for every bounded predictable process ϕ : for a suitable Brownian motion B defined on an eventual extension of the probability space.
The following theorem gives us the continuity of the controlled dynamics with respect to the control variable in the sense of law.Theorem 3.12 Let μ be a relaxed control and X μ be the corresponding relaxed process.Assume that the relaxed SDE (3.5) has a unique weak solution.Then there exists a sequence (u n ) of strict controls such that the sequence (X u n ) converges in law to X μ .
Proof According to the Chattering lemma, there exists (u n ) of strict controls such that dtδ u n t (da) converges weakly to μ t (da) dt, P-a.s.Let (X u n ) and X μ be the solutions of (3.5) corresponding to dtδ u n t (da) and dtμ t (da).
Since b and σ are bounded, it follows that Therefore (X u n ) is tight on the space D([0, T], R d ).Since (dtδ u n t (da)) converges weakly to dtμ s (da) and M n (dt, da) = δ u n s (da) dB s converges to M(dt, da), and according to the uniqueness in law of the relaxed SDE (3.5), it holds that (X u n ) converges in law to X μ .We will prove under pathwise uniqueness that the approximation holds in quadratic mean.Theorem 3. 13 Let μ be a relaxed control, and let X be the solution of (3.5).Assume that the coefficients of stochastic differential equation (2.1) are continuous and bounded.Assume also that pathwise uniqueness holds for (3.5).Then there exists a sequence ii) There exists a subsequence u n k such that J u n k converges to J(μ), (3.6) where X n denotes the solution of the stochastic differential equation (3.5) associated with (u n ).
Proof i) Let μ be a relaxed control, then by Lemma 3.10 there exists a sequence (u n ) such that μ n t (da) dt = δ u n t (da) dt − → μ t (da) dt in R, P-a.s.Let X n and X be the solutions of (3.5) associated with μ n and μ, respectively.Suppose that the result of Theorem 3.13 is false, then there exists γ > 0 such that According to the compactness of A and the boundedness of the coefficients of SDE (3.5), it follows that the family of processes is tight.Then, by the Skorokhod selection theorem [21], there exist a probability space ( , F, P) and a sequence n = ( μ n , υ n , X n , Y n , M n , N n ) defined on it such that: i) For each n ∈ N, the laws of n and n coincide; ii) There exists a subsequence ( n k ), still denoted by of n , which converges to , Pa.s., where = ( μ, υ, X, Y , M, N).By the uniform integrability, we have where E is the expectation with respect to P. According to i), we see that X n and Y n satisfy the following equations: Since ( n ) converges to , Pa.s., ( X n ) and ( Y n ) converge respectively to X and Y , which satisfy According to Lemma 3.10, the sequence (μ n , μ) converges to (μ, μ) in R 2 .Moreover, Hence, law( μ, υ) = law(μ, μ), which implies that μ = υ, Pa.s.By the same method, we can prove that M = N , Pa.s.According to the pathwise uniqueness of equation (3.5), it follows that X = Y , Pa.s., which contradicts (3.7).i) is proved.
ii) This is a direct consequence of i) along with the continuity and boundedness of the functions h and g.Remark 3.14 1) Using the same arguments, we can replace the sequence (δ u n t (da) dt) by any sequence (μ n t (da) dt) of relaxed controls converging weakly to μ t (da) dt.This means in particular that the function μ − → X μ is continuous.
2) As a consequence of the last theorem, the value functions of the strict and relaxed control problems are equal.Therefore, by relaxing the control problem, the value function remains unchanged.Moreover, the relaxed control problem has an optimal solution.

Discussion of another relaxed model
Assume that both the drift and the diffusion coefficients are controlled.Let us consider another type of relaxation of the controlled stochastic differential equation, suggested in the literature by many authors [1,6,35].Instead of relaxing the infinitesimal generator of the controlled process, the authors considered the direct relaxation of the stochastic differential equation as in deterministic control.This is carried out by integrating directly the drift and diffusion coefficient against the relaxed control, which gives the following equation: This "relaxed" form has the advantage to be linear with respect to the control variable, with a convex compact set of controls.However, its solution has a serious drawback in that it is not continuous with respect to the control variable.As a consequence, it follows that the value functions of the strict and relaxed control problems cannot be equal.Therefore, it cannot be considered as a true relaxed model.Moreover we have no mean to prove the existence of an optimal relaxed control as the dynamics and the cost functional are not continuous with respect to the control variable.
Indeed, consider the control problem governed by the following SDE: where admissible controls are measurable functions u : [0, 1] → A ={-1, 1}.The corresponding "relaxed" equation is defined by (3.9) Proposition 3. 15 The solution of the controlled SDE (3.9) is not continuous in the control variable.
Proof Consider the sequence of Rademacher functions According to Lemma 3.1, the sequence of relaxed controls (dt.δu n (t) (da)) converges weakly to dt. 1  2 (δ -1 + δ 1 )(da).Let X n t be the solution of SDE (3.9) associated with the relaxed control dt.δ u n (t) (da).It is clear that is a continuous martingale with quadratic variation X n , X n t = t 0 u 2 n (s).ds = t.Therefore (X n t ) is a Brownian motion constructed possibly on an augmented probability space.Let X * be the relaxed state process corresponding to the limit μ * = dt. 1 2 (δ -1 + δ 1 )(da), then It is obvious that the sequence of state processes (X n t ) does not converge in any topology to Remark 3.16 1) It is clear that the right limit is a Brownian motion, which could be represented as X * (t) = t 0 A a.M(ds, da) where M(dt, da) = 2 i=1 1 2 dB i s 1 (α i ∈da) , where B 1 and B 2 are independent Brownian motions constructed possibly on an augmentation of the probability space.
2) As a consequence of the last proposition, the value functions of the strict and "relaxed" control problems could be different.Moreover, even if the set V is compact, there is no mean to prove the existence of an optimal control for this model.
3) Unlike the model based on SDE (3.5), the controlled stochastic equation (3.8) is driven by the martingale measure μ t (da) dB t , which is not orthogonal.Its intensity is given by μ t (da) ⊗ μ t (da) ⊗ dt.This is a worthy martingale measure in the sense of Walsh [34].

Necessary conditions for optimality
We know from the previous section that an optimal relaxed control μ exists in the set R. This implies the existence of a filtered probability space still denoted by ( , F, (F t ) t≥0 , P), a measure-valued control dtμ t (da), and an orthogonal martingale measure M(da, dt) whose covariance measure is dtμ t (da) such that: and Our goal in this section is to derive necessary conditions for optimality satisfied by the optimal relaxed control μ.According to the Chattering lemma, dtμ t (da) can be approximated in the sense of weak convergence by a sequence (u n ) of strict controls.We start by establishing the necessary conditions of near optimality that are satisfied by the strict controls (u n ).This important auxiliary result is based on Ekeland's variational principle [13] and is interesting in itself.Indeed in most practical situations it is sufficient to characterize and compute nearly optimal controls.Lemma 4.1 (Ekeland's variational principle) Let (E, d) be a complete metric space and f : E → R be lower semicontinuous and bounded from below.Given ε > 0, suppose that u ε ∈ E satisfies f (u ε ) ≤ inf(f ) + ε.Then, for any λ > 0, there exists ν ∈ E such that Let us endow the set U ad of strict controls with an appropriate metric.For any u and v ∈ U ad , we set where P ⊗ dt is the product measure of P with the Lebesgue measure dt.Remark 4.2 It is well known that (U ad , d) is a complete metric space and that the cost functional J is continuous from U ad into R, see [26].Now, let μ ∈ R be an optimal relaxed control and denote by X the solution of (4.1) controlled by μ.From Lemma 3.10 and Theorem 3.13, there exists a sequence (u n ) of strict controls such that where X n is the solution of (3.5) corresponding to μ n = δ u n t (da) dt.Let us introduce the usual Hamiltonian of the system H(t, x, u, p, q, r) = b(t, x, u), p + tr q * σ (t, x, u)h(t, x, u), where A * denotes the transpose of the vector or matrix A.
We define as in [36] by (p, q) and (P, Q) the first-and second-order adjoint processes satisfying the following backward SDEs, assuming that (F t ) is the natural filtration of the Brownian motion: where b x (t) = b x (t, X t , u t ) and σ j x (t) = σ j x (t, X t , u t ) and h x (t) = h x (t, X t , u t ).Under conditions (A 1 ) and (A 2 ), BSDEs (4.3) and (4.4) have unique solutions satisfying the following estimates: Remark 4.3 In case (F t ) is not necessarily the natural filtration of the Brownian motion, we must add in the backward equations (4.3) and (4.4) two cadlag martingales that are orthogonal to the Brownian motion.This comes from the Itô representation theorem for Brownian martingales.

Necessary conditions for near optimality
The generalized Hamiltonian H associated with a strict control u and the corresponding state process X is defined as in [36] by where (p, q) and (P, Q) are solutions of the adjoint equations (4.3) and (4.4).The following theorem gives necessary conditions for near optimality for the strict control u n in terms of an approximate maximum principle.See [38] Theorem 4.1 for a complete proof of this intermediary result.
Proposition 4.4 There exists a sequence of strict controls (u n ) such that and there exist unique adapted solutions (p n , q n ) and (P n , Q n ) of the adjoint equations (4.3) and (4.4), corresponding to the admissible pair (u n , X n ), such that for any γ ∈ [0, 1/3) Proof Let us give the outline of the proof.According to the optimality of μ and the Chattering lemma, there exist a sequence (ε n ) of positive numbers with lim n→∞ ε n = 0 and a sequence of strict controls (u n ) such that (u n ) According to a suitable version of Lemma 4.1 with λ = ε 2 3 , Let us define the perturbation From (4.6) we have Using the definition of d, it holds that 0 ≤ J u n,h -J u n + ε Let us denote by x n,h the solution of (2.1) corresponding to u n,h , which is defined in (4.7).
To get the desired variational inequality we differentiate the function J(u n,h ) with respect to h at h = 0. See [38] Theorem 4.1 for details.

The relaxed maximum principle
Let X be the corresponding optimal state process associated with the optimal relaxed control μ, and (p, q) and (P, Q) be the solutions of the adjoint equations (4.9) and (4.10) associated with (μ, X).We assume that (F t ) is the natural filtration of the Brownian motion.
where k = k(t, X t , μ t ) = A k(t, X t , a)μ t (da), and k stands to be b x , σ x , f x , h x , and H xx .
Theorem 4.5 (Relaxed maximum principle) Assume (A 1 ) and (A2).Let (μ, X) be an optimal pair, then there exist unique adapted solutions (p, q) and (P, Q) of the adjoint equations (4.9) and (4.10), respectively, such that The proof of this theorem is based on the following stability theorem of adjoint processes with respect to the control variable.Theorem 4.6 (Stability theorem for BSDEs) Let (p n , q n ), (P n , Q n ), and (resp.(p,q), (P, Q)) be the solutions of (4.3) and (4.4) associated with the pair (u n , X n ) (resp the solutions of (4.9) and (4.10) associated with the pair (μ, X).Then we have i)

.13)
Proof i) Let us write down the drivers of the first-order adjoint equations (4.3) and (4.9) corresponding to (u n , X n ) and (μ, X).
where Let us deal with the first term on the right-hand side of (4.14).
b x being Lipschitz in x and (X n t ) converges to X t uniformly in t in probability imply that the first term on the right-hand side of (4.15) converges in probability to 0.
In addition, we have E(sup 0≤t≤T |p(t)| 2 ) < +∞, therefore sup 0≤t≤T |p(t)| < +∞, P-a.s, which implies the existence of a P-negligible set N such that for each ω / ∈ N there exist In particular, for each ω / ∈ N , the function b x (t, X t , E(X t ), a)p(t).1 [0,t] is a measurable bounded function in (t, a) and continuous in a; therefore it is a test function for the stable convergence.Hence, by using the fact that (δ u n t (da) dt) converges in V to μ t (da) dt, P-a.s., it follows that the second term on the right-hand side tends to 0, P-a.s.
The other terms containing p(t) can be handled by using the same techniques.The terms in (4.14) containing q(t) can be treated similarly.However, one should pay a little more attention as q(t) is just square integrable (in (t, ω)).More precisely, The first integral on the right-hand side may be treated by using similar arguments as previously as the function (σ n x (t)σ x (t))q(t)1 {|q(t)|≤N} is measurable bounded and continuous in a.The second term tends to 0 by Chebyshev's inequality using the square integrability of q(t).
ii) is proved by using similar arguments.
Proof of Theorem 4.5.The main result is proved by passing to the limit in inequality (4.5) and using Theorem 4. Therefore passing at the limit as h tends to zero, we obtain for any ε not in some Lebesgue null set N .The last inequality is true for all B ∈ F ε, , then for any bounded F ε -measurable random variable F it holds that E FH(ε, X ε , μ ε ) ≥ E FH(ε, X ε , υ) , which leads to We conclude by using the measurability of the Hamiltonian with respect to F ε .

Example
To illustrate our results, we present an example inspired from [36].To simplify the notations, we suppose that the problem is one dimensional.Assume that the dynamics is given by ⎧ ⎨ ⎩ dx t = u(t) dB t , t ∈ [0, 1] x 0 = 0 (4.19) and the cost functional is defined by The strict controls are measurable functions from [0, 1] to the set {-1, 1}.By replacing x t = 1 0 u(s) dB s in the cost functional, we obtain Since ( 3 2t) > 0 for any t ∈ [0, 1], it is clear that J(u) attains its minimum for u(t) = 0, with the state process x(t) = 0.But this is impossible as the strict controls take only the values -1 and 1.
The cost functional associated with a relaxed control is then defined by The cost functional attains its minimum at α(t) = 1 2 and the optimal control is given by μ = dt 1 2 δ 1 (da) + 1 2 δ -1 (da) .
Let us verify that this optimal control satisfies the necessary conditions of Theorem 4.5.
The first-and second-order adjoint processes (p t , q t ) and (P t , Q t ) are the unique adapted solutions of first-and second-order adjoint equations.The unique solutions are (p t , q t ) = (0, 0) and (P t , Q t ) = (2t -4, 0).

Theorem 3 . 5
Under assumption (A 1 ), the relaxed optimal control problem defined by the martingale problem (3.3) and the relaxed cost functional(3.4)admits an optimal solution.
Proof For any g continuous in [0, T] × A, suppose that μ(t, da) has continuous sample paths.Let n ≥ 1, and let (T i = [t i , s i [) be subintervals of the interval [0, T] of length not exceeding 2 -n .Cover A by finitely many disjoint sets (A j ) such that diameter (A j ) ≤ 2 -n .Choose a point (t i , a ij ) in T i × A j .We have j μ(t i , A j ) = 1.Subdivide each T i further into disjoint left-closed, right-open intervals T ij such that its length is the product of μ(t i , A j ) with the length of T Define the sequence of predictable process μ n (•) by μ n (t, da) = δ a ij (da) for t ∈ T ij .And by path-continuity of u(•), we may increase n further if necessary to obtain da) dt converges weakly to μ t (da) dt P-a.s.i .∀ε > 0, for n large enough, we haveg(t, a)g(t i , a ij ) < ε for (t, a) ∈ T i × A j , sup a g(t, a)g(t i , a) < ε for t ∈ T i .
[20]a μ(t, da) where f stands for b x , σ x , h x .By using the result of Hu and Peng[20], Theorem 2.1, it is sufficient to show that n (t, p t , q t ) -G(t, p t , q t ) dt n (t, p t , q t ) -G(t, p t , q t ) dt ≤ is obvious.Let us prove the inequality from the other sense.If υ ∈ P(A) is a probability measure on A, then Remark Since P(A) is a subspace of V whose elements are constant (in (ω, t)) relaxed controls, then (4.16) may be replaced by (Pontryagin's relaxed maximum principle).Under the same conditions as in Theorem 4.5, there exists a Lebesgue negligible subset N in the interval [0, T] such that for any t not in N it holds thatH(t, X t , μ t ) = sup Proof Let ε ∈]0, T[ and B ∈ F ε ,for small h > 0 define the relaxed control 7to get the desired inequality(4.11).Corollary 4.7Under the same conditions as in Theorem 4.5 it holds thatE T 0 H(t, X t , μ) dt = sup t , υ) = A H(t, X t ,a)υ(da) and P(A) is the space of probability measures on A.Proof Since {δ a (da); a ∈ A} ⊂ P(A), it is clear that the inequalitysup υ∈P(A) E T 0 H(t, X t , υ) ≥ sup a∈A E T 0 H(t, X t , a) t , υ) dt ∈ conv E T 0 H(t, X t , a) dt , a ∈ A , T 0 H(t, X t , υ) dt ≤ sup a∈A E T 0 H(t, X t , a) dt .B for ε < t < ε + h μ t otherwise,where υ is a probability measure on A. It follows from (4.16) that 1/h ε+h ε E 1 B H(t, X t , μ t ) dt ≥ 1/h θ+h θ E 1 B H(t, X t , υ) dt.