Suppose there are n players in an ongoing competition, with player i having value vi, and suppose that a game between i and j is won by i with probability vi/(vi + vj). Consider the winner plays competition where in each stage two players play a game, and the winner keeps playing in the next game. We consider two models for choosing its opponent, analyze both models as Markov chains, and determine their stationary probabilities as well as other quantities of interest.

Computationally, when we solve for the stationary probabilities for a countable-state Markov chain, the transition probability matrix of the Markov chain has to be truncated, in some way, into a finite matrix. Different augmentation methods might be valid such that the stationary probability distribution for the truncated Markov chain approaches that for the countable Markov chain as the truncation size gets large. In this paper, we prove that the censored (watched) Markov chain provides the best approximation in the sense that, for a given truncation size, the sum of errors is the minimum and show, by examples, that the method of augmenting the last column only is not always the best.

A particular random walk on the integers leads to a new, tractable Markov chain. Of the stationary probabilities, we discuss the existence, some analytic properties and a factorization which leads to an algorithmic procedure for their numerical computation. We also consider the positive recurrence of some variants which each call for different mathematical arguments. Analogous results are derived for a continuous version on the positive reals.

This paper considers a certain class of continuous-time Markov processes, whose time-dependent and stationary distributions are studied. In the stationary case, the analogy with Whittle's relaxed Markov process is pointed out. The derivation of the probability generating functions of the general process provides useful results for the analysis of some population and queueing processes.