A novel iterative algorithm for solving coupled Riccati equations. (English) Zbl 1433.93156

Summary: In this paper, a novel iterative algorithm is developed for solving the coupled algebraic Riccati equation arising from the quadratic optimal control problem for continuous-time Markovian jump linear systems. First, two existing iterative algorithms to solve the coupled Riccati matrix equation are reviewed. Next, based on analysis for these two algorithms, a new iterative algorithm that combines both the information in the current iterative step and the information in the last iterative step is proposed. It is shown that the proposed algorithm with proper initial conditions can monotonically converge to the unique positive definite solution of the coupled Riccati matrix equation if the associated Markovian jump system is stochastically stabilizable. Also, numerical examples show that the presented algorithm is faster than some previous algorithms when the weighted parameter is appropriately selected.


93E20 Optimal stochastic control
49K45 Optimality conditions for problems involving randomness
49N10 Linear-quadratic optimal control problems
60J28 Applications of continuous-time Markov processes on discrete state spaces
65H10 Numerical computation of solutions to systems of equations
93D15 Stabilization of systems by feedback
93E03 Stochastic systems in control theory (general)
Full Text: DOI


[1] Shi, P.; Boukas, E. K., \(H_∞\)-control for Markovian jumping linear systems with parametric uncertainty, J. Optim. Theory Appl., 95, 75-99 (1997) · Zbl 1026.93504
[2] Fragoso, M. D.; Costa, O. L., A separation principle for the continuous-time LQ-problem with Markovian jump parameters, IEEE Trans. Autom. Control, 55, 2692-2707 (2010) · Zbl 1368.93785
[3] Costa, E. F.; Val, J. B.D., On the observability and detectability of continuous-time Markov jump linear systems, SIAM J. Control Optim., 41, 1295-1314 (2002) · Zbl 1029.93007
[4] Mukaidani, H.; Hua, X.; Koichi, M., Recursive algorithm for mixed \(H_2/H_∞\) control problem of singularly perturbed systems, Int. J. Syst. Sci., 3, 1, 1299-1312 (2000) · Zbl 1080.93586
[5] Song, J.; He, S.; Liu, F.; Niu, Y.; Ding, Z., Data-driven policy iteration algorithm for optimal control of continuous-time \(It \hat{o}\) stochastic systems with Markovian jumps, IET Control Theory Appl., 10, 1431-1439 (2016)
[6] Denman, E. D., An additional algorithm for a system of coupled algebraic matrix Riccati equations, IEEE Trans. Comput., 25, 91-93 (1976) · Zbl 0317.93038
[7] Salama, A. I.A.; Gourishankar, V., A computational algorithm for solving a system of coupled algebraic matrix Riccati equations, IEEE Trans. Comput., 23, 100-102 (1974) · Zbl 0298.65060
[8] Benner, P.; Byers, R., An exact line search method for solving generalized continuous-time algebraic Riccati equations, IEEE Trans. Autom. Control, 43, 101-107 (1998) · Zbl 0908.93026
[9] Do, V. J.B. R.; Geromel, J. C.; Costa, O. L.V., Solutions for the linear-quadratic control problem of Markov jump linear systems, J. Optim. Theory Appl., 103, 283-311 (1999) · Zbl 0948.49018
[10] Gajic, Z.; Borno, I., Lyapunov iterations for optimal control of jump linear systems at steady state, IEEE Trans. Autom. Control, 40, 1971-1975 (1995) · Zbl 0837.93073
[11] Gajic, Z.; Losada, R., Monotonicity of algebraic Lyapunov iterations for optimal control of jump parameter linear systems, Syst. Control Lett., 41, 175-181 (2000) · Zbl 0985.93017
[12] Ivanov, I.; Imsland, L.; Bogdanova, B., Iterative algorithms for computing the feedback Nash equilibrium point for positive systems, Int. J. Syst. Sci., 48, 729-737 (2017) · Zbl 1362.49022
[13] Ivanov, I. G., On some iterations for optimal control of jump linear equations, Nonlinear Anal. Theory Methods Appl., 69, 4012-4024 (2008) · Zbl 1162.65020
[14] Wu, A. G.; Duan, G. R., New iterative algorithms for solving coupled Markovian jump Lyapunov equations, IEEE Trans. Autom. Control, 60, 289-294 (2015) · Zbl 1360.65105
[15] Wu, A. G.; Duan, G. R.; Liu, W. Q., Implicit iterative algorithms for continuous Markovian jump Lyapunov equations, IEEE Trans. Autom. Control, 61, 3183-3189 (2015) · Zbl 1359.65061
[16] Rami, M. A.; Xun, Y. Z., Linear matrix inequalities, Riccati equations, and infinite stochastic linear quadratic controls, IEEE Trans. Autom. Control, 45, 1131-1143 (2000) · Zbl 0981.93080
[17] Ji, Y.; Chizeck, H. J., Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Autom. Control, 35, 777-788 (1990) · Zbl 0714.93060
[18] Wonham, W. M., On a matrix Riccati equation of stochastic control, SIAM J. Control, 6, 681-697 (1968) · Zbl 0182.20803
[19] Bibby, J., Axiomatisations of the average and a further generalisation of monotonic sequences, Glasg. Math. J., 15, 63-65 (1974) · Zbl 0291.40003
[20] Wu, A. G.; Sun, H. J.; Zhang, Y., An SOR implicit iterative algorithm for coupled Lyapunov equations, Automatica, 91, 38-47 (2018) · Zbl 1406.93310
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.