Estimation of hidden Markov models for a partially observed risk sensitive control problem. (English) Zbl 1274.93254

Summary: This paper provides a summary of our recent work on the problem of combined estimation and control of systems described by finite state, hidden Markov models. We establish the stochastic framework for the problem, formulate a separated control policy with risk-sensitive cost functional, describe an estimation scheme for the parameters of the hidden Markov model that describes the plant, and finally indicate how the combined estimation and control problem can be reformulated in a framework that permits an application of stochastic approximation techniques to the proof of asymptotic convergence of the estimator.


93E10 Estimation and detection in stochastic control theory
Full Text: Link


[1] Arapostathis A., Marcus S. I.: Analysis of an identification algorithm arising in the adaptive estimation of Markov chains. Mathematics of Control, Signals and Systems 3 (1990),1-29 · Zbl 0685.93063
[2] Baras J. S., James M. R.: Robust and Risk-Sensitive Output Feedback Control for Finite State Machines and Hidden Markov Models, to be publishe. · Zbl 0911.93055
[3] Benveniste A., Métivier M., Priouret P.: Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, Berlin 1990. Translation of “Algorithmes adaptatifs et approximations stochastiques”, Masson, Paris 1987 · Zbl 0752.93073
[4] Fernandéz-Gaucherand E., Marcus S. I.: Risk-Sensitive Optimal Control of Hidden Markov Models: Structural Results. Technical Report TR 96-79, Institute for Systems Research, University of Maryland, College Park, Maryland 1996 · Zbl 0891.93087
[5] Fernandéz-Gaucherand E., Arapostathis A., Marcus S. I.: Analysis of an adaptive control scheme for a partially observed controlled Markov chain. IEEE Trans. Automat. Control 38 (1993), 6, 987-993 · Zbl 0786.93089
[6] Krishnamurthy, V, Moore J. B.: On-line estimation of hidden Markov model parameters based on the. IEEE Trans. Signal Processing 41 (1993), 8, 2557-2573 · Zbl 0825.93742
[7] Gland F. Le, Mevel L.: Geometric Ergodicity in Hidden Markov Models. Technical Report No. 1028, IRISA/INRIA, Campus de Beaulieu, Renees 1996 · Zbl 0941.93053
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.