Risk-sensitive optimal control. (English) Zbl 0718.93068

Whittle, Peter

Wiley-Interscience Series in Systems and Optimization. Chichester etc.: John Wiley & Sons, Inc. x, 246 p. $ 59.95 (1990).

This work has two major themes. One is that of risk-sensitive control, in that the quadratic cost function of the standard LQG (linear/quadratic/Gaussian) treatment is replaced by the exponential of a quadratic, giving the so-called LEQG formulation. The effect of the generalization is to provide an extra tempering parameter, which can be said to set the level of the optimizer’s degree of optimism or pessimism - i.e. his degree of belief that random events will work out to his advantage or disadvantage.
Formally viewed, the extension is very interesting. All the familiar LQG theory has an LEQG analogue, although a reorientation of ideas is sometimes needed before one can see it. Once one has achieved this reorientation, then the LQG (‘risk-neutral’) theory appears, not only as a special case but almost as a degenerate one. For example, the risk- sensitive certainty-equivalence principle yields exactly the stochastic maximum principle, couched wholly in terms of observable and computable quantities, which has been sought for years. Also, as has been realized by Glover and Doyle, the \(H_{\infty}\) and minimum entropy criteria, which have awoken so much interest in recent years, amount exactly to an infinite horizon version (more exactly, an average-optimal version) of LEQG theory, despite the great difference in starting point.
The second theme is the so-called path-integral or Hamiltonian formulation. The use of recursive methods of optimization reduces the optimal control problem to one of solving a Riccati equation. However, it has been known for years that there is an elegant formulation of optimal LQ control wholly in terms of linear operators, a canonical operator factorization replacing the solution of a Riccati equation. This approach generalizes naturally to higher-order (non-Markov) models. However, it also covers the LEQG formulation in the most natural way. Appeal to the risk-sensitive certainty-equivalence principle leads to a restatement of the control-optimization problem as the extremization of a quadratic path integral. The linear stationarity equations have an appealing Hamiltonian form, and the only entry in the matrix operator of this system which is vacant in the LQ case is neatly filled by the risk-sensitivity parameter.

MSC:

93E20	Optimal stochastic control
93-02	Research exposition (monographs, survey articles) pertaining to systems and control theory
93B35	Sensitivity (robustness)
91A60	Probabilistic games; gambling

Keywords:

risk-sensitive control; quadratic cost function; LQG theory; stochastic maximum principle; recursive methods of optimization; Riccati equation