Indefinite quadratic with linear costs optimal control of Markov jump with multiplicative noise systems. (English) Zbl 1115.49021

Summary: In this paper we consider the stochastic optimal control problem of discrete-time Markov jump with multiplicative noise linear systems. The performance criterion is assumed to be formed by a linear combination of a quadratic part and a linear part in the state and control variables. The weighting matrices of the state and control for the quadratic part are allowed to be indefinite. We present a necessary and sufficient condition under which the problem is well posed and a state feedback solution can be derived from a set of coupled generalized Riccati difference equations interconnected with a set of coupled linear recursive equations. For the case in which the quadratic-term matrices are non-negative, this necessary and sufficient condition can be written in a more explicit way. The results are applied to a problem of portfolio optimization.


49K45 Optimality conditions for problems involving randomness
49K40 Sensitivity, stability, well-posedness
93E20 Optimal stochastic control
91B28 Finance etc. (MSC2000)
60J05 Discrete-time Markov processes on general state spaces
Full Text: DOI


[1] Ait Rami, M.; Chen, X.; Moore, J. B.; Zhou, X. Y., Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls, IEEE Transactions on Automatic Control, 46, 428-440 (2001) · Zbl 0992.93097
[2] Ait Rami, M.; Chen, X.; Zhou, X. Y., Discrete-time indefinite LQ control with state and control dependent noises, Journal of Global Optimization, 23, 245-265 (2002) · Zbl 1035.49024
[3] Ait Rami, M.; Moore, J. B.; Zhou, X. Y., Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM Journal on Control Optimization, 40, 1296-1311 (2001) · Zbl 1009.93082
[4] Ait Rami, M.; Zhou, X. Y., Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls, IEEE Transactions on Automatic Control, 45, 1131-1143 (2000) · Zbl 0981.93080
[5] Basin, M.; Perez, J.; Skliar, M., Optimal filtering for polynomial system states with polynomial multiplicative noise, International Journal of Robust and Nonlinear Control, 16, 303-314 (2006) · Zbl 1105.93056
[6] Basin, M.; Perez, J.; Skliar, M., Optimal state filtering and parameter identification for linear systems, (Proceedings of the 2006 American control conference (2006), Minneapolis: Minneapolis Minnesota), 987-990
[7] Beghi, A.; D’Alessandro, D., Discrete-time optimal control with control-dependent noise and generalized Riccati difference equations, Automatica, 34, 1031-1034 (1998) · Zbl 0944.93032
[8] Blair, W. P.; Sworder, D. D., Feedback control of a class of linear discrete systems with jump parameters and quadratic cost criteria, International Journal of Control, 21, 833-841 (1975) · Zbl 0303.93084
[9] Chen, S.; Li, X.; Zhou, X. Y., Stochastic linear quadratic regulators with indefinite control weight costs, SIAM Journal on Control Optimization, 36, 1685-1702 (1998) · Zbl 0916.93084
[10] Costa, O. L.V.; Fragoso, M. D.; Marques, R. P., Discrete-time Markov jump linear systems (2005), Springer: Springer Berlin · Zbl 1081.93001
[11] Costa, O. L.V.; Kubrusly, C. S., State-feedback \(H_\infty \)-control for discrete-time infinite-dimensional stochastic bilinear systems, Journal of Mathematical Systems, Estimation Control, 6, 1-32 (1996) · Zbl 0844.93036
[12] Dombrovskii, V. V.; Dombrovskii, D. V.; Lyashenko, E. A., Predictive control of random-parameter systems with multiplicative noise. Application to investment portfolio optimization, Automation and Remote Control, 66, 583-595 (2005) · Zbl 1114.93099
[13] Dombrovskii, V. V.; Lyashenko, E. A., A linear quadratic control for discrete systems with random parameters and multiplicative noise and its application to investment portfolio optimization, Automatic and Remote Control, 64, 1558-1570 (2003) · Zbl 1061.93097
[14] Dragan, V.; Morozan, T., Stability and robust stabilization to linear stochastic systems described by differential equations with Markovian jumping and multiplicative white noise, Stochastic Analysis and Applications, 20, 33-92 (2002) · Zbl 1136.60335
[15] Dragan, V.; Morozan, T., The linear quadratic optimization problems for a class of linear stochastic systems with multiplicative white noise and Markovian jumping, IEEE Transactions on Automatic Control, 49, 665-675 (2004) · Zbl 1365.93541
[16] Gershon, E.; Shaked, U., Static \(H_2\) and \(H_\infty\) output-feedback of discrete-time LTI systems with state multiplicative noise, Systems & Control Letters, 55, 232-239 (2006) · Zbl 1129.93372
[17] Li, D.; Ng, W.-L., Optimal dynamic portfolio selection: Multiperiod mean-variance formulation, Mathematical Finance, 10, 387-406 (2000) · Zbl 0997.91027
[18] Li, X.; Zhou, X. Y., Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2, 265-282 (2002) · Zbl 1119.93418
[19] Li, X.; Zhou, X. Y.; AitRami, M., Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27, 149-175 (2003) · Zbl 1031.93155
[20] Lim, A.; Zhou, X. Y., Stochastic optimal control LQR control with integral quadratic constraints and indefinite control weights, IEEE Transactions on Automatic Control, 44, 1359-1369 (1999) · Zbl 0970.93038
[21] Liu, Y.; Yin, G.; Zhou, X. Y., Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41, 1063-1070 (2005) · Zbl 1091.93019
[22] Luo, C.; Feng, E., Generalized differential Riccati equation and indefinite stochastic LQ control with cross term, Applied Mathematics and Computation, 155, 121-135 (2004) · Zbl 1053.93041
[23] Moore, J. B.; Zhou, X. Y.; Lim, A. E.B., Discrete-time LQG controls with control dependent noise, Systems & Control Letters, 36, 199-206 (1999) · Zbl 0913.93076
[24] Ran, A. C.M.; Trentelman, H. L., Linear quadratic problems with indefinite cost for discrete time systems, SIAM Journal on Matrix Analysis and Applications, 14, 776-797 (1993) · Zbl 0786.93043
[25] Roll, R., A mean/variance analysis of tracking error, Journal of Portfolio Management, 18, 13-22 (1992)
[26] Rudolf, M.; Wolter, H. J.; Zimmermann, H., A linear model for tracking error minimization, Journal of Banking and Finance, 23, 85-103 (1999)
[27] Saberi, A.; Sannuti, P.; Chen, B. M., \(H_2\)-Optimal control (1995), Prentice-Hall: Prentice-Hall Englewood Cliffs, NJ · Zbl 0876.93001
[28] Wu, H.; Zhou, X. Y., Characterizing all optimal controls for an indefinite stochastic linear quadratic control problem, IEEE Transactions on Automatic Control, 47, 1119-1122 (2002) · Zbl 1364.49044
[29] Yin, G.; Zhou, X. Y., Markowitz’s mean-variance portfolio selection with regime switching: From discrete-time models to their continuous-time limits, IEEE Transactions on Automatic Control, 49, 349-360 (2004) · Zbl 1366.91148
[30] Zhou, X. Y.; Li, D., Continuous-time mean-variance portfolio selection: A stochastic LQ framework, Applied Mathematics and Optimization, 42, 19-33 (2000) · Zbl 0998.91023
[31] Zhou, X. Y.; Yin, G., Markowitz’s mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42, 1466-1482 (2003) · Zbl 1175.91169
[32] Zhu, J., On stochastic Riccati equations for the stochastic LQR problem, Systems & Control Letters, 54, 119-124 (2005) · Zbl 1129.93549
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.