- Research Article
- Open Access
- Published:
Infinite Horizon Discrete Time Control Problems for Bounded Processes
Advances in Difference Equations volume 2008, Article number: 654267 (2009)
Abstract
We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal control problems for bounded processes, for systems governed by difference equations. Results due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary conditions with weakened hypotheses of concavity and without invertibility, and we provide new results on the adjoint variable. We show links between bounded problems and nonbounded ones. We also give sufficient conditions of optimality.
1. Introduction
The first works on infinite horizon optimal control problems are due to Pontryagin and his school [1]. They were followed by few others [2–6]. We consider in this paper an infinite horizon Optimal Control problem in the discrete time framework. Such problems are fundamental in the macroeconomics growth theory [7–10] and see references of [11]. Even in the finite horizon case, the discrete time framework presents significant differences from the continuous time one. Boltianski [12] shows that in the discrete time case, a convexity condition is needed to guarantee a strong Pontryagin Principle while this last one can be obtained without such condition in the continuous time setting. We study our problem in the space of bounded sequences which allows us to use Analysis in Banach spaces instead of using reductions to finite horizon problems as in [5, 6]. According to Chichlinisky [13, 14], the space of bounded sequences was first used in economics by Debreu [15]. It can also be found in [7, 8, 16]. We obtain Pontryagin Maximum Principles in the strong form using weaker convexity hypotheses than the traditional ones and without invertibility [5]. When we study the problem in a general sequence space it turns out that the infinite series will not always converge. Therefore we present other notions of optimality that are currently used, notably in the economic literature, see [3, 4, 9] and we show how our problem can be related to these other problems. We end the paper by establishing sufficient conditions of optimality.
Now we briefly describe the contents of the paper. In Section 2 we introduce the notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary conditions of optimality namely the existence of the adjoint variable in the space satisfying the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove these theorems through some lemmas and using results due to Ioffe-Tihomirov [17]. In Section 4 we introduce some other notions of optimality for problems in the nonbounded case and we show links with our problem. For example, we show that when the objective function is positive then a bounded solution is a solution among the unbounded processes. Finally we give sufficient conditions of optimality for problems in the bounded and unbounded cases adapting for each case the approprate transversality condition.
2. Pontryagin maximum principles for bounded processes
We first precise our notations. Let be a nonempty open convex subset of
and
a nonempty compact subset of
Let
and, for all
We set
Define
For every define
as the closure of the set of terms of the sequence
If
is compact. We set
such that
is thus the set of the bounded sequences which are in the interior of
Note that
is a convex open subset of
since
is open and convex. We set
Define
; it is the set of admissible processes with respect to the considered dynamical system.
Let We consider first the following problem (P1):

which can be written as follows.
(P1) Maximize when
Theorem 2.1.
Let be a solution of (P1). Assume the following.
-
(i)
For all
the mapping
is of class
on
and for all
the mapping
is Fréchet-differentiable on
-
(ii)
For all
for all
for all
for all
there exists
such that
(2.1)
-
(iii)
For any compact set
there exists a constant
such that for all
for all
for all
and
-
(iv)
There exists
such that
and for all
(2.2)
Then there exists such that
-
(a)
-
(b)
-
(c)
Comments
For continuous time problems, one does not need conditions to obtain a strong Pontryagin maximum principle, both in the finite horizon case (see, e.g., [18]) and in the infinite horizon case (see, e.g., [5]). But for discrete time problems, strong Pontryagin principles cannot hold without an additional assumption namely a convexity condition, as Boltyanski shows in [12] for the finite horizon framework. Condition (ii) comes from the Ioffe and Tihomirov book [17]. It generalizes the usual convexity condition used to garantee a strong Pontryagin maximum principle. The usual condition is: convex subset,
concave with respect to
and for every
affine with respect to
It implies condition (ii). In (iii) the condition
is satisfied when
is continuous (since
is compact) and the condition
is satisfied when
exists and is continuous.
Conclusion (a) is the adjoint equation, conclusion (b) is the strong Pontryagin maximum principle and conclusion (c) is a transversality condition at infinity. In our case (c) is immediately obtained since is in
but in general (nonbounded cases) it is very delicate to obtain such a conclusion. [9]
In the next theorem we consider the autonomous case. Thus the hypotheses are simpler and easier to manipulate.
Theorem 2.2.
Let for all
Let
be a solution of (P1). Assume that the following conditions are fulfilled.
-
(i)
For all
the mappings
and
are of class
on
-
(ii)
For all
for all
for all
for all
there exists
such that
(2.3)
-
(iii)
Then there exists such that the assertions (a), (b), and (c) of Theorem 2.1 are satisfied.
3. Proofs of Theorems 2.1 and 2.2
Proof of Theorem 2.1
First Part
The first part of the proof goes through several lemmas.
Lemma 3.1.
is well-defined and under hypothesis (i) of Theorem 2.1, for all
the mapping
is of class
and one has, for all
For the proof see [19].
We set for all
Lemma 3.2.
Assume that hypothesis (iii) of Theorem 2.1 holds. Then for one has
Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold, then for all
the mapping
is of class
on the ball
in
and for all
one has
Proof.
Let Let
be the constant of (iii) with
So for all
So one has
Assume now that hypotheses (i) and (iv) of Theorem 2.1 hold. Let us show that is of class
on
Take
Let
be given. Let
be such that
Then, for all
under (iv), which implies that
Let us now show that is Fréchet-differentiable on
Take
Let
be given. Let
be such that
Then, for all
under (iii). But this implies that
Thus
is Fréchet-differentiable at
and
To show the continuity of at
let
be the constant of hypothesis (iii) corresponding to
Let
be given and let
be such that
So
is of class
Lemma 3.3.
Under hypothesis (ii) of Theorem 2.1, for all for all
for all
there exists
such that

Proof.
Let and
Hypothesis (ii) of Theorem 2.1 implies for all
the existence of
such that

Therefore we obtain

Set so
and satisfies the required relations.
Lemma 3.4.
Under hypotheses (i) and (iv) of Theorem 2.1,
Proof.
Since the problem is a problem of bounded solutions of first-order linear difference equations.
Let Assume that
Then for all
there exists a unique
such that for all

where
Consider the operator such that for all

where

Recall that the norm of
is defined by
and that the norm of a linear operator
between normed spaces is defined by
So So
Since
and
is invertible so it is surjective.
Set Then under (iv) one has
So
is surjective that is
Recall that where
consists of all singular functionals, see Aliprantis and Border [20]. In fact it consists (up to scalar multiples) of all extensions of the "limit functional" to
If then there exists
such that for all
(
being the space of convergent sequences having a limit in
)
Lemma 3.5 ().
If then
where
for every
So there exists
such that for all
Second Part
Our optimal control problem can be written as the following abstract static optimisation problem in a Banach space:

that satisfies all conditions of Theorem 4.3, Ioffe-Tihomirov [17]. So we can apply this theorem and obtain the existence of not all zero,
such that:

(AE) denotes the adjoint equation of this problem and (PMP) the Pontryagin maximum principle. They can be written, respectively:

Set where
and
(AE) becomes:

So we get

Let be arbitrarily chosen in
and let
be in
Consider the sequence
defined as follows:

So one has if
hence
Thus, it holds that
Now
Therefore, for all and for all
one has

which implies

that is,

(PMP) becomes:

So for all
Consider, for all the sequences
defined as follows:

Since the inequality holds for every we obtain

using as
is of finite support.
Lemma 3.6.
().
Proof.
Recall we obtained the existence of not all zero,
such that:

If then
since
Hence We can set it equal to one.
From Lemma 3.6 and the previous results, conclusions (a) and (b) are satisfied.
Conclusion (c) is a straightforward consequence of the belonging of to
Lemma 3.7.
().
Proof.
Indeed we obtained for all
Using
one has
for all
Thus
Proof of Theorem 2.2.
Define on
such that
Under hypothesis (i) of Theorem 2.2, for all
the mappings
and
are of class
on
The proof can be found in [19].
We consider the proof of Lemma 3.4 and we set Then the proof goes like that of Theorem 2.1.
4. Results for unbounded problems
We study now problems of maximization over admissible processes which are not necessarily bounded when the optimal solution is bounded. So consider the following problems.
(P2) Maximize on
(P3) Find such that, for all
(P4) Find such that, for all
(P5) Find such that, for all
The optimality notion of (P3) is called "the strong optimality," that of (P4) is called "the overtaking optimality" and that of (P5) the "weak overtaking optimality" in [3] (in the continuous-time framework). Many existence results of overtaking optimal solutions and weakly overtaking optimal solutions are obtained in [3, 4]. In [4] there are also results in the discrete-time framework.
Remark 4.1.
Notice that is an optimal solution of (P3) implies
is an optimal solution of (P4) which implies
is an optimal solution of (P5).
Moreover if is a bounded optimal solution of (P4) then (P3) and (P4) reduce to the same problem.
Lemma 4.2.
The two following assertions hold.
-
(a)
If
is an optimal solution of problem (P2), (P3), (P4) or (P5) and
then
is an optimal solution of problem (P1). Therefore Theorem 2.1 applies.
-
(b)
Assume
on
If
is an optimal solution of problem (P3) or (P4) and
then
Proof. (a) Since a bounded optimal solution of (P2) or (P3) is an optimal solution of (P1). Suppose now that
is a bounded optimal solution of (P4) that is
for all
Since
this can be written
for all
and so in particular for all
In that case The proof is analogous for (P5).
(b) If is an optimal solution of problem (P3) and
one has
for all
Since
the sequence
is increasing and since it is also upper bounded it converges in
So and
Theorem 4.3.
Let for all
One assumes the following conditions fulfilled:
-
(i)
on
-
(ii)
For all
there exists
Then one has
-
(a)
-
(b)
If
is an optimal solution of problem (P1), then it is an optimal solution of problems (P3), (P4), and (P5) which all reduce to the same problem.
Remark 4.4. (b) shows that under a nonnegativity assumption, solving the problem in the space of bounded processes provides solutions for problems in spaces of admissible processes which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny [21] where problems are studied in the continuous-time case.
Proof. (a) It is clear that the following inequality holds:

Let Let
be given and let
be such that
Set

where is such that
and
are bounded and
Since on
one has

so we obtain

Since this is true fo all letting
we obtain

(b) Since for all
the sequence
is nonnegative and nondecreasing so it converges in
So Hence (P3), (P4) reduce to the same problem. Similarly (P5) reduces to it. Let
be an optimal solution of problem (P1) and suppose it is not an optimal solution of problem (P3). So there exists
such that
that is

Let and
Construct
and
as in (a). Thus
Hence we obtain so
which contradicts the hypothesis so
is an optimal solution of problem (P3).
Following Michel, [22], for all and for all
we define
as the set of the
for which there exists
satisfying
We also define
as the set of the
for which there exists
satisfying
for all
Theorem 4.5.
Let for all
Let
be an optimal solution of problem (P1). One assumes the following conditions fulfilled.
-
(i)
on
-
(ii)
For all
there exists
-
(iii)
For all
the mappings
and
are Fréchet-differentiable at
-
(iv)
For all
for all
co
where co denotes the convex hull.
-
(v)
For all
is invertible.
Then there exists such that
and
-
(a)
-
(b)
for all
for all
Remark 4.6.
Notice that condition (iv) is a convexity condition and that condition (ii) of Theorem 2.2 implies this condition (iv). Condition (ii) of Theorem 2.2 is equivalent to the following condition: for all the set
is convex.
Proof.
Use Theorem 4.3 of this paper and apply Theorem 3 in Blot-Chebbi [5].
5. Sufficient conditions of optimality
Let for all
Theorem 5.1.
Let where
is convex. One assumes that there exists
and that the following conditions are fulfilled.
-
(i)
The mappings
and for all
are of class
on
-
(ii)
-
(iii)
-
(iv)
The mapping
is concave with respect to
for all
Then is an optimal solution of (P1).
Proof.
Notice that from (ii), for all
Let For all
one has

therefore, we obtain

Since for all is concave with respect to
and
one has
Using hypothesis (iii) with
gives
and using hypothesis (iii), the first order necessary condition for the optimality of
is
Thus one has
The hypothesis implies
and since
and
belong to
one has
Hence we obtain
so
That is
Corollary 5.2.
Let (resp.,
). If the hypotheses of the previous theorem are satisfied except that
is replaced by
and if the following hypothesis is also satisfied:
-
(v)
then is a solution of (P3) (resp., (P4)).
Notice that if with
we obtain that
is a solution of (P5).
One can weaken the hypothesis of concavity of with respect to
and
and replace it by the concavity of
with respect to
as the following theorem shows. (See [23] for a quick survey of sufficient conditions.)
Let
The maximum is attained since is compact.
Theorem 5.3.
Let One assumes that there exists
and that the following hypotheses are fulfilled.
-
(i)
For all
the mappings
and for all
are of class
on
-
(ii)
Also (iii) of the previous theorem.
-
(iv)
The mapping
is concave with respect to
for all
Then is an optimal solution of (P1).
Proof.
Let and let
For all
one has

by the definition of and noticing that
So we obtain

(Notice that Now using
(see Seierstad and Sydsaeter [24, page 390]) we obtain
The concavity of
with respect to
gives
Finally
follows as in the proof of the previous theorem.
Corollary 5.4.
Let (resp.,
). If the hypotheses of the previous theorem are satisfied except that
is replaced by
and if the following hypothesis is also satisfied:
-
(v)
then is a solution of (P3) (resp., (P4)).
Notice that if with
we obtain that
is a solution of (P5).
References
Pontryagin L, Boltianski V, Gramkrelidze R, Mitchenko E: Théorie Mathématique des Processus Optimaux. Mir, Moscow, Russia; 1974.
Ekeland I, Scheinkman JA: Transversality conditions for some infinite horizon discrete time optimization problems. Mathematics of Operations Research 1986, 11(2):216-229. 10.1287/moor.11.2.216
Carlson DA, Haurie AB, Leizarowitz A: Infinite Horizon Optimal Control. Deterministic and Stochastic Systems. 2nd edition. Springer, Berlin, Germany; 1991.
Zaslavski AJ: Turnpike Properties in the Calculus of Variations and Optimal Control, Nonconvex Optimization and Its Applications. Volume 80. Springer, New York, NY, USA; 2006:xxii+395.
Blot J, Chebbi H: Discrete time Pontryagin principles with infinite horizon. Journal of Mathematical Analysis and Applications 2000, 246(1):265-279. 10.1006/jmaa.2000.6797
Blot J, Hayek N: Infinite-horizon Pontryagin principles with constraints. In Communications of the Laufen Colloquium on Science, Laufen, Austria, Berichte aus der Mathematik, 2. Edited by: Ruffing A, Suhrer A, Suhrer J. Shaker, Aachen, Germany; 2007:1-14.
Araujo A, Scheinkman JA: Smoothness, comparative dynamics, and the turnpike property. Econometrica 1977, 45(3):601-620. 10.2307/1911677
Araujo A, Scheinkman JA: Notes on comparative dynamics. In General Equilibrium, Growth, and Trade. Edited by: Green JR, Scheinkman JA. Academic Press, New York, NY, USA; 1979:217-226.
Michel P: Some clarifications on the transversality condition. Econometrica 1990, 58(3):705-723. 10.2307/2938197
Stokey NL, Lucas RE Jr., Prescott EC: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, Mass, USA; 1989:xviii+588.
Grinold R: Convex infinite horizon programs. Mathematical Programming 1983, 25(1):64-82. 10.1007/BF02591719
Boltianski V: Commande Optimale des Systèmes Discrets. Traduit du Russe par A. Sossinsk. Mir, Moscow, Russia; 1976:467.
Chichilnisky G: An axiomatic approach to sustainable development. Social Choice and Welfare 1996, 13(2):231-257. 10.1007/BF00183353
Chichlinisky G: What is sustainable development? Land Economics 1997, 73(4):46791.
Debreu G: Valuation equilibrium and Pareto optimum. Proceedings of the National Academy of Sciences of the United States of America 1954, 40(7):588-592. 10.1073/pnas.40.7.588
Peleg B, Ryder H: On optimal consumption plans in a multi-sector economy. Review of Economics Studies 1972, 39(2):159-169. 10.2307/2296868
Ioffe AD, Tihomirov VM: Theory of Extremal Problems, Studies in Mathematics and Its Applications. Volume 6. North-Holland, Amsterdam, The Netherlands; 1979:xii+460.
Alexéev V, Tikhomirov V, Fomine S: Commande Optimale. Mir, Moscow, Russia; 1982:447.
Blot J, Crettez B: On the smoothness of optimal paths. Decisions in Economics and Finance 2004, 27(1):1-34. 10.1007/s10203-004-0042-5
Aliprantis CD, Border KC: Infinite-Dimensional Analysis. 2nd edition. Springer, Berlin, Germany; 1999:xx+672.
Blot J, Cartigny P: Optimality in infinite-horizon variational problems under sign conditions. Journal of Optimization Theory and Applications 2000, 106(2):411-419. 10.1023/A:1004611816252
Michel P: Programmes mathématiques mixtes. Application au principe du maximum en temps discret dans le cas déterministe et dans le cas stochastique. RAIRO Recherche Opérationnelle 1980, 14(1):1-19.
Blot J, Hayek N: Sufficient conditions for infinite-horizon calculus of variations problems. ESAIM: Control, Optimisation and Calculus of Variations 2000, 5: 279-292.
Seierstad A, Sydsaeter K: Sufficient conditions in optimal control theory. International Economic Review 1977, 18(2):367-391. 10.2307/2525753
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Blot, J., Hayek, N. Infinite Horizon Discrete Time Control Problems for Bounded Processes. Adv Differ Equ 2008, 654267 (2009). https://doi.org/10.1155/2008/654267
Received:
Accepted:
Published:
DOI: https://doi.org/10.1155/2008/654267
Keywords
- Transversality Condition
- Adjoint Equation
- Pontryagin Maximum Principle
- Adjoint Variable
- Finite Horizon