The general framework of a Markov chain in a random environment is presented and the problem of determining extinction probabilities is discussed. An efficient method for determining absorption probabilities and criteria for certain absorption are presented in the case that the environmental process is a two-state Markov chain. These results are then applied to birth and death, queueing and branching chains in random environments.

The paper examines those continuous time Markov processes Z(·) on the positive integers which have the ‘skip free upwards’ property, with regard to their asymptotic behaviour in the event of Z(t) tending to infinity. The behaviour is characterised in terms of the convergence or divergence of an appropriate function of Z(t), and the description is improved by central limit and iterated logarithm theorems. The conditions of the theorems are expressed entirely in terms of the matrix Q of instantaneous transition rates for Z(·). The method is applied, by way of example, to the super-critical linear birth and death process.

A simple birth-death model of particle fluctuations is studied where at each discrete time a birth and/or death may occur. We show that if the probability of a birth does not depend on the number of particles present and if births and deaths are independent, then the times between successive deaths are independent geometrically distributed random variables, which is false in the general case. Since the above properties of the times between successive neuron firings have been observed in nerve cells, the model proposed in [2] obtains added credence.