×
Costa, O. L. V.; Aya, J. C. C.

Temporal difference methods for the maximal solution of discrete-time coupled algebraic Riccati equations. (English) Zbl 0984.93051

J. Optimization Theory Appl. 109, No. 2, 289-309 (2001).
The authors present an iterative technique for deriving the maximal solution of a set of discrete-time coupled algebraic Riccati equations, based on temporal difference methods. They trace a parallel with the theory of temporal difference algorithms for Markovian decision processes to develop a \(\lambda\)-policy iteration like algorithm for the maximal solution of these equations. The advantage of the proposed method is that an appropriate choice of \(\lambda\) between 0 and 1 can speed up the convergence of the policy evaluation step of the policy iteration method by using value iteration.
Reviewer: Jihong Dou (Xian)

Cited in 7 Documents

MSC:

93C55 Discrete-time control/observation systems
93C40 Adaptive control/observation systems
65F30 Other matrix algorithms (MSC2010)
49N10 Linear-quadratic optimal control problems
93B40 Computational methods in systems theory (MSC2010)

Keywords:

maximal solution; discrete-time coupled algebraic Riccati equations; temporal difference methods; Markovian decision processes; \(\lambda\)-policy iteration
PDF BibTeX XML Cite
\textit{O. L. V. Costa} and \textit{J. C. C. Aya}, J. Optim. Theory Appl. 109, No. 2, 289--309 (2001; Zbl 0984.93051)
Full Text: DOI
  • logo of hbz
  • logo of WorldCat

References:

[1] Mariton, M., Jump Linear Systems in Automatic Control, Marcel Dekker, New York, NY, 1990.
[2] Costa, O. L. V., and Fragoso, M. D., Discrete-Time LQ-Optimal Control Problems for Infinite Marko Jump Parameter Systems, IEEE Transactions on Automatic Control, Vol. 40, pp. 2076–2088, 1995. · Zbl 0843.93091
[3] Ji, Y., and Chizeck, H. J., Controllability, Observability, and Discrete-Time Markovian Jump Linear Quadratic Control, International Journal of Control, Vol. 48, pp. 481–498, 1988. · Zbl 0669.93007
[4] Ji, Y., Chizeck, H. J., Feng, X., and Loparo, K. A., Stability and Control of Discrete-Time Jump Linear Systems, Control Theory and Advanced Technology, Vol. 7, pp. 247–270, 1991.
[5] Abou-Kandil, H., Freiling, G., and Jank, G., On the Solution of Discrete-Time Markovian Jump Linear-Quadratic Control Problems, Automatica, Vol. 31, pp. 765–768, 1995. · Zbl 0822.93074
[6] Rami, M. A., and El Ghaoui, L., LMI Optimization for Nonstandard Riccati Equations Arising in Stochastic Control, IEEE Transactions on Automatic Control, Vol. 41, pp. 1666–1671, 1996. · Zbl 0863.93087
[7] Costa, O. L. V., Do Val, J. B. R., and Geromel, J. C., A Convex Programming Approach to 2 -Control of Discrete-Time Markovian Jump Linear Systems, International Journal of Control, Vol. 66, pp. 557–579, 1997. · Zbl 0951.93536
[8] Do Val, J. B. R., Geromel, J. C., and Costa, O. L. V., Uncoupled Riccati Iterations for the Linear-Quadratic Control Problem of Discrete-Time Markov Jump Linear Systems, IEEE Transactions on Automatic Control, Vol. 43, pp. 1727–1733, 1998. · Zbl 1056.93537
[9] Do Val, J. B. R., Geromel, J. C., and Costa, O. L. V., Solution for the Linear-Quadratic Control Problem of Marko Jump Linear Systems, Journal of Optimization Theory and Applications, Vol. 103, pp. 283–311, 1999. · Zbl 0948.49018
[10] Gajic, Z., and Borno, I., Lyapuno Iterations for Optimal Control of Jump Linear Systems at Steady State, IEEE Transactions on Automatic Control, Vol. 40, pp. 481–498, 1995. · Zbl 0837.93073
[11] Costa, O. L. V., and Boukas, E. K., Necessary and Sufficient Condition for Robust Stability of Continuous-Time Linear Systems with Markovian Jumps, Journal of Optimization Theory and Applications, Vol. 99, pp. 359–379, 1998. · Zbl 0919.93082
[12] Bertsekas, D. P., and Tsitsilklis, J.N., Neurodynamic Programming, Athena Scientific, Belmont, Massachusetts, 1996.
[13] Sutton, R. S., and Barto, A. G., Reinforcement Learning: An Introduction, MIT Press, Cambridge, Massachusetts, 1998.
[14] Costa, O. L. V., and Fragoso, M. D., Stability Results for Discrete-Time Linear Systems with Markovian Jumping Parameters, Journal of Mathematical Analysis and Applications, Vol. 179, pp. 154–178, 1993. · Zbl 0790.93108
[15] Mariton, M., Almost Sure and Moment Stability of Jump Linear Systems, Systems and Control Letters, Vol. 11 pp. 393–397, 1988. · Zbl 0672.93073
[16] Costa, O. L. V., and Marques, R. P., Maximal and Stabilizing Hermitian Solutions for Discrete-Time Coupled Algebraic Riccati Equations, Mathematics of Control Signals and Systems, Vol. 12, pp. 167–195, 1999. · Zbl 0928.93047
[17] Blair, W. P., Jr., and Sworder, D. D., Feedback Control of a Class of Linear Discrete System with Jump Parameters and Quadratic Cost Criteria, International Journal of Control, Vol. 21, pp. 833–841, 1975. · Zbl 0303.93084
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.