Epilogue

Summary

We conclude by highlighting a string of issues that still remain open.

1. In the controlled martingale problem with ergodic cost, we obtained existence of an optimal ergodic process and optimal Markov process separately, but not of an optimal ergodic Markov process, as one would expect from one's experience with the nondegenerate case. This issue still remains open. In particular it is unclear whether the Krylov selection procedure of Section 6.7, which has been used to extract an optimal Markov family for the discounted cost problem under nondegeneracy, can be similarly employed for the ergodic problem. The work in Bhatt and Borkar [22] claims such a result under very restrictive conditions, but the proof has a serious flaw.

2. The HJB equation was analyzed in two special cases. The general case remains open. In particular, experience with discrete state space problems gives some pointers:

1. (a) In the multichain case for Markov chains with finite state space S and finite action space A, a very general dynamic programming equation is available due to Howard [53], viz.,

2. for i ∈ S. Here the unknowns are the value function V and the state dependent optimal cost ϱ. An analog of this for the degenerate diffusion case could formally be written down as

3. This has not been studied.

4. […]