In this article we consider the ergodic risk-sensitive control problem for a large class of multidimensional controlled diffusions on the whole space. We study the minimization and maximization problems under either a blanket stability hypothesis, or a near-monotone assumption on the running cost. We establish the convergence of the policy improvement algorithm for these models. We also present a more general result concerning the region of attraction of the equilibrium of the algorithm.

We present a new iterative procedure for solving the multiple stopping problem in discrete time and discuss the stability of the algorithm. The algorithm produces monotonically increasing approximations of the Snell envelope which coincide with the Snell envelope after finitely many steps. Unlike backward dynamic programming, the algorithm allows us to calculate approximative solutions with only a few nestings of conditional expectations and is, therefore, tailor-made for a plain Monte Carlo implementation.

Formulae are presented for the variance and higher moments of the present value of single-stage rewards in a finite Markov decision process. Similar formulae are exhibited for a semi-Markov decision process. There is a short discussion of the obstacles to using the variance formula in algorithms to maximize the mean minus a multiple of the standard deviation.