This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. We give conditions on the game's primitive data under which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using the extended infinitesimal operator associated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide a recursive way of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.