Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n)P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n)P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

The xlogx condition is a fundamental criterion for the rate of growth of a general branching process, being equivalent to non-degeneracy of the limiting random variable. In this paper we adopt the ideas from Lyons, Pemantle and Peres (1995) to present a new proof of this well-known theorem. The idea is to compare the ordinary branching measure on the space of population trees with another measure, the size-biased measure.

A continuous-time Markov chain on the non-negative integers is called skip-free to the right (left) if only unit increments to the right (left) are permitted. If a Markov chain is skip-free both to the right and to the left, it is called a birth–death process. Karlin and McGregor (1959) showed that if a continuous-time Markov chain is monotone in the sense of likelihood ratio ordering then it must be an (extended) birth–death process. This paper proves that if an irreducible Markov chain in continuous time is monotone in the sense of hazard rate (reversed hazard rate) ordering then it must be skip-free to the right (left). A birth–death process is then characterized as a continuous-time Markov chain that is monotone in the sense of both hazard rate and reversed hazard rate orderings. As an application, the first-passage-time distributions of such Markov chains are also studied.

Let ζ be a Markov chain on a finite state space D, f a function from D to ℝd, and Sn = ∑k=1nf(ζk). We prove an invariance theorem for S and derive an explicit expression of the limit covariance matrix. We give its exact value for p-reinforced random walks on ℤ2 with p = 1, 2, 3.

Previous work in extending Wald's equations to Markov random walks involves finiteness of moment generating functions and uniform recurrence assumptions. By using a new approach, we can remove these assumptions. The results are applied to establish finiteness of moments of ladder variables and to derive asymptotic expansions for expected first passage times of Markov random walks. Wiener–Hopf factorizations for Markov random walks are also applied to analyse ladder variables and related first passage problems.

This paper introduces a new stochastic process in which the iterates of a dynamical system evolving in discrete time coincide with the events of a Poisson process. The autocovariance function of the stochastic process is studied and a necessary and sufficient condition for it to vanish is deduced. It is shown that the mean function of this process comprises a continuous-time semidynamical system if the underlying dynamical map is linear. The flow of probability density functions generated by the stochastic process is analysed in detail, and the relationship between the flow and the solutions of the linear Boltzmann equation is investigated. It is shown that the flow is a semigroup if and only if the point process defining the stochastic process is Poisson, thereby providing a new characterization of the Poisson process.

Given a sequence of random variables (rewards), the Haviv–Puterman differential equation relates the expected infinite-horizon λ-discounted reward and the expected total reward up to a random time that is determined by an independent negative binomial random variable with parameters 2 and λ. This paper provides an interpretation of this proven, but previously unexplained, result. Furthermore, the interpretation is formalized into a new proof, which then yields new results for the general case where the rewards are accumulated up to a time determined by an independent negative binomial random variable with parameters k and λ.

We present new closed form solutions for partition functions used to normalize the steady-state flow balance equations of certain Markovian type queueing networks. The results focus on single class closed product form networks with state space constraints at the queueing stations. They are achieved by combining the partition function of the open network, having finite local buffers with a delta function in order to fix the number of customers in the system.

We prove a central limit theorem for conditionally centred random fields, under a moment condition and strict positivity of the empirical variance per observation. We use a random normalization, which fits non-stationary situations. The theorem applies directly to Markov random fields, including the cases of phase transition and lack of stationarity. One consequence is the asymptotic normality of the maximum pseudo-likelihood estimator for Markov fields in complete generality.

Is the Ewens distribution the only one-parameter family of partition structures where the total number of types sampled is a sufficient statistic? In general, the answer is no. It is shown that all counterexamples can be generated via an urn scheme. The urn scheme need only satisfy two general conditions. In fact, the conditions are both necessary and sufficient. However, in particular, for a large class of partition structures that naturally arise in the infinite alleles theory of population genetics, the Ewens distribution is the only one in this class where the total number of types is sufficient for estimating the mutation rate. Finally, asymptotic sufficiency for parametric families of partition structures is discussed.

We introduce a new class of interacting particle systems on a graph G. Suppose initially there are Ni(0) particles at each vertex i of G, and that the particles interact to form a Markov chain: at each instant two particles are chosen at random, and if these are at adjacent vertices of G, one particle jumps to the other particle's vertex, each with probability 1/2. The process N enters a death state after a finite time when all the particles are in some independent subset of the vertices of G, i.e. a set of vertices with no edges between any two of them. The problem is to find the distribution of the death state, ηi = Ni(∞), as a function of Ni(0).

We are able to obtain, for some special graphs, the limiting distribution of Ni if the total number of particles N → ∞ in such a way that the fraction, Ni(0)/S = ξi, at each vertex is held fixed as N → ∞. In particular we can obtain the limit law for the graph S2, the two-leaf star which has three vertices and two edges.

Size-biased permutation (SBP) is a random arrangement of frequencies of distinct categories in the order in which the categories appear for the first time in the sampling process. We study the conditions under which the SBPs converge in distribution and discuss extended versions of SBP for the case when the sum of positive frequencies is less than 1.

We first study an epidemic amongst a population consisting of individuals with the same infectivity but with varying susceptibilities to the disease. The asymptotic final epidemic size is compared with the corresponding size for a homogeneous population. Then we group a heterogeneous population into households, assuming very high infectivity within households, and investigate how the global infection pressure is affected by rearranging individuals between the households. In both situations considered, it turns out that whether or not homogenizing the individuals or households will result in an increased spread of infection actually depends on the infectiousness of the disease.

The time until extinction for the closed SIS stochastic logistic epidemic model is investigated. We derive the asymptotic distribution for the extinction time as the population grows to infinity, under different initial conditions and for different values of the infection rate.

The distribution of the final size, K, in a general SIR epidemic model is considered in a situation when the critical parameter λ is close to 1. It is shown that with a ‘critical scaling’ λ ≈ 1 + a / n1/3, m ≈ bn1/3, where n is the initial number of susceptibles and m is the initial number of infected, then K / n2/3 has a limit distribution when n → ∞. It can be described as that of T, the first passage time of a Wiener process to a parabolic barrier b + at − t2/2. The proof is based on a diffusion approximation. Moreover, it is shown that the distribution of T can be expressed analytically in terms of Airy functions using the spectral representation connected with Airy's differential equation.

An interacting particle system modelling competing growth on the ℤ2 lattice is defined as follows. Each x ∈ ℤ2 is in one of the states {0,1,2}. 1's and 2's remain in their states for ever, while a 0 flips to a 1 (a 2) at a rate equal to the number of its neighbours which are in state 1 (2). This is a generalization of the well-known Richardson model. 1's and 2's may be thought of as two types of infection, and 0's as uninfected sites. We prove that if we start with a single site in state 1 and a single site in state 2, then there is positive probability for the event that both types of infection reach infinitely many sites. This result implies that the spanning tree of time-minimizing paths from the origin in first passage percolation with exponential passage times has at least two topological ends with positive probability.

For fixed i let X(i) = (X1(i), …, Xd(i)) be a d-dimensional random vector with some known joint distribution. Here i should be considered a time variable. Let X(i), i = 1, …, n be a sequence of n independent vectors, where n is the total horizon. In many examples Xj(i) can be thought of as the return to partner j, when there are d ≥ 2 partners, and one stops with the ith observation. If the jth partner alone could decide on a (random) stopping rule t, his goal would be to maximize EXj(t) over all possible stopping rules t ≤ n. In the present ‘multivariate’ setup the d partners must however cooperate and stop at the same stopping time t, so as to maximize some agreed function h(∙) of the individual expected returns. The goal is thus to find a stopping rule t* for which h(EX1 (t), …, EXd(t)) = h (EX(t)) is maximized. For continuous and monotone h we describe the class of optimal stopping rules t*. With some additional symmetry assumptions we show that the optimal rule is one which (also) maximizes EZt where Zi = ∑dj=1Xj(i), and hence has a particularly simple structure. Examples are included, and the results are extended both to the infinite horizon case and to the case when X(1), …, X(n) are dependent. Asymptotic comparisons between the present problem of finding suph(EX(t)) and the ‘classical’ problem of finding supEh(X(t)) are given. Comparisons between the optimal return to the statistician and to a ‘prophet’ are also included. In the present context a ‘prophet’ is someone who can base his (random) choice g on the full sequence X(1), …, X(n), with corresponding return suph(EX(g)).

We consider an allocation of n balls into N cells according to probabilities pi. Assuming that the balls are allocated successively, denote by φ(n,N) the number of such balls which go into an already occupied cell. If n = 2 the probability that two balls will occupy the same cell is equal to the so-called match probability MP = p21 + … + p2N. An upper estimate for the probability ℙ(φ(n,N) ≤ m) which depends only on n and MP is derived. Such inequalities are important for estimation of the reliability of DNA fingerprinting, a new method of crime investigation which is currently much debated.

Very little is known about the quantitative behaviour of dynamical systems with random excitation, unless the system is linear. Known techniques imply the resolution of parabolic partial differential equations (Fokker–Planck–Kolmogorov equation), which are degenerate and of high dimension and for which there is no effective known method of resolution. Therefore, users (physicists, mechanical engineers) concerned with such systems have had to design global linearization techniques, known as equivalent statistical linearization (Roberts and Spanos (1990)). So far, there has been no rigorous justification of these techniques, with the notable exception of the paper by Kozin (1987). In this contribution, using large deviation principles, several mathematically founded linearization methods are proposed. These principles use relative entropy, or Kullback information, of two probability measures, and Donsker–Varadhan entropy of a Gaussian measure relatively to a Markov kernel. The method of ‘true linearization’ (Roberts and Spanos (1990)) is justified.

The ‘minimal’ repair of a system can take several forms. Statistical or black box minimal repair at failure time t is equivalent to replacing the system with another functioning one of the same age, but without knowledge of precisely what went wrong with the system. Its major attribute is its mathematical tractability. In physical minimal repair, at system failure time t, we minimally repair the ‘component’ which brought the system down at time t. The work of Arjas and Norros, Finkelstein, and Natvig is reviewed. The concept of a rate function for minimal repairs of the statistical and physical types are discussed and developed. It is shown that the number of physical minimal repairs is stochastically larger than the number of statistical minimal repairs for k out of n systems with similar components. Some majorization results are given for physical minimal repair for two component parallel systems with exponential components.