Several simple stochastic models are given for a finite closed system of individuals existing in clusters which may come together to form larger clusters which may in turn split up. Some of these models are analysed and compared in equilibrium. Several of the models fit into a general framework established and investigated by Whittle (1965a); it is shown that these models have an equilibrium solution of a particularly simple form, deduced by Whittle, if and only if the models are stochastically reversible. A normal approximation to two of the models in equilibrium is found to give the same mean value as a deterministic approximation.

Criteria are established for a discrete-time Markov process {Xn}n≧0 in Rd to have strictly positive, respectively zero, probability of escaping to infinity. These criteria are mainly in terms of the mean displacement vectors μ(y) = E{Xn+1|Xn = y} – y, and are essentially such that they force a deterministic process w.p.1 to move off to infinity, respectively to return to a compact set infinitely often. As an application we determine of most two-dimensional birth and death processes with rates linearly dependent on the population, whether they can escape to infinity or not.

Let Z(t) be the population size at time t in a general age-dependent branching process (as defined by Crump and Mode, or Jagers) in which the number N of offspring of a parent has expected value 1 (critical case). Assuming positivity and finiteness of the second moments of N, of the lifespan distribution and of the expected number of births per parent as a function of age (also assumed to be strongly non-lattice), the distribution of Z(t)/t conditioned on non-extinction at time t is asymptotically exponential. The main step in the proof is a comparison lemma for the probability generating functions of Z(t) and of the embedded generation process.

This paper is concerned with a discrete Markov process related to an underlying weighted graph. With this graph we associate an urn containing p types of balls which correspond to the p vertices of the graph, and such that the number of balls of each type is proportional to the weight of the related vertex. A drawing scheme from the urn is defined which leads to Markovian non-homogeneous transition probabilities. Many kinds of weighted graphs are found to have a strong convergence property; this is that the number of balls drawn from the urn at the end of any single draw converges in probability to the total number of balls in the urn, as this tends to infinity.

A stochastic model for a periodic screening program is presented in which the natural history of a chronic disease is assumed to follow a progressive path from a preclinical state to a clinical state. The sampling of preclinical state sojourn times by screening examinations generates bounds on the preclinical state recurrence times. The distribution of the bounded forward recurrence time is derived and used to obtain the distribution and mean of the lead time, and relationships for calculating the proportion of preclinical cases detected. These expressions are derived in terms of the preclinical state sojourn-time distribution and adjustible parameters important in the design of a periodic screening program.

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.

A discrete-time Markov decision model with a denumerable set of states and unbounded costs is considered. It is shown that the optimality equation of dynamic programming along with some additional, easily checked, conditions may be used to establish the optimality or ∊ -optimality of policies with respect to the average expected cost criterion. The results are used to derive optimal policies in two queueing examples.