Some concepts of dependence have recently been introduced by Ebrahimi (1987) to explore the structural properties of the hitting times of bivariate processes. In this framework, the special case of univariate processes has curious features. New properties are derived for this case. Some applications to sequential inference and inequalities for Brownian motion and new better than used (NBU) processes are also provided.

Prompted by a rivulet model for the flow of liquid through packed columns we consider a simple random walk on parallel axes moving at different rates. A particle may make one of three transitions at each time instant: to the right or to the left on the axis it was on at the previous time instant, or across to the other axis. Results are obtained for the unrestricted walk, and for the walk with absorbing, or reflecting, end-points.

Let W (s, t), s, t ≧ 0, be the two-parameter Yeh–Wiener process defined on the first quadrant of the plane, that is, a Gaussian process with independent increments in both directions. In this paper, a lower bound for the distribution of the supremum of W (s, t) over a rectangular region [0, S]×[0, T], for S, T > 0, is given. An upper bound for the same was known earlier, while its exact distribution is still unknown.

Let ∊1∊2, · ··, be a stationary sequence satisfying the weak long-range dependence condition Δ (un(τ)) of [3] for every τ > 0, where nP(∊1> un(τ))→ τ. Assume only that P (there are j exceedances of un(τ) by ∊1, ∊2, · ··, ∊n) converges for all j with 0≦j≦υ<∞ and a given fixedτ. Then the same holds for every τ> 0. For 0≦j≦υ the limit is P(X = j) where X is compound Poisson and the multiplicity distribution is independent ofτ. These results are extended to more general levels un and to cases where the joint distribution of the numbers of exceedances of several levels is considered. The limiting distributions of linearly normalized extreme order statistics are derived as a corollary. An application to insurance claim data is discussed.

A sequence of finite point processes {Pn} is constructed in using a Markov sequence of points. Essentially, in the process Pn consisting of n events, the coordinates of these events are simply the first n points of a Markov sequence suitably scaled so that the average density of the process is independent of n. The second-order properties of Pn are discussed and sufficient conditions are found for Pn to converge in distribution to a Poisson process as n →∞. A simple example involving the cardioid distribution is described.

A model for the service time structure in the single server queue is given embodying correlations between contiguous and near-contiguous service times. A number of results are derived in the case of Poisson arrivals both for equilibrium and the transient state. In particular, Kendall's (equilibrium) result P (a departure leaves the queue empty) = 1 — (mean service time)/(mean inter-arrival time) is found still to hold good. The effect of the correlation on the mean and variance of the equilibrium queue length distribution is examined in a simple case.

We study weighted ensemble, an interacting particle method for sampling distributions of Markov chains that has been used in computational chemistry since the 1990s. Many important applications of weighted ensemble require the computation of long time averages. We establish the consistency of weighted ensemble in this setting by proving an ergodic theorem for time averages. As part of the proof, we derive explicit variance formulas that could be useful for optimizing the method.

We give a very general reformulation of multi-actor Markov decision processes and show that there is a tendency for the actors to take the same action whenever possible. This considerably reduces the complexity of the problem, either facilitating numerical computation of the optimal policy or providing a basis for a heuristic.

The goal of this paper is to provide formulae for the expectation and variance of the height and length of the ancestral recombination graph (ARG). While the formula for the expectation of the height is known (see, e.g. Krone and Neuhauser (1997)), the other formulae seem to be new. We obtain in particular (see Theorem 4.1) a very simple formula which expresses the expectation of the length of the ARG as a linear combination of the expectations of both the length of the coalescent tree and the height of the ARG. Finally, we study the speed at which the ARG comes down from infinity.

This work concerns Markov decision chains with finite state spaces and compact action sets. The performance index is the long-run risk-sensitive average cost criterion, and it is assumed that, under each stationary policy, the state space is a communicating class and that the cost function and the transition law depend continuously on the action. These latter data are not directly available to the decision-maker, but convergent approximations are known or are more easily computed. In this context, the nonstationary value iteration algorithm is used to approximate the solution of the optimality equation, and to obtain a nearly optimal stationary policy.

A Markov chain model of a population consisting of a finite or countably infinite number of colonies with N particles at each colony is considered. There are d types of particle and transition from the nth generation to the (n + 1)th is made up of three stages: reproduction, migration, and sampling. Natural selection works in the reproduction stage. The limiting diffusion operator (as N→∞) for the proportion of types at colonies is found. Convergence to the diffusion is proved under certain conditions.

In the literature on mixed Poisson processes, a number of characterisation properties have been studied. As a new characterisation property for mixed Poisson processes, we show that normalised event occurrence times are the order statistics of independent uniform random variables on (0,1). Berman's theorem on lp-isotropic sequences is applied to prove the results.

In the Poisson disorder problem the probability that the parameter of a Poisson process y, has increased by a constant amount, given observations of ys, s ≦ t, is a Markov process of mixed type with jumps of variable (state-dependent) magnitude superimposed upon a drift which satisfies an ordinary differential equation. Using a likelihood-ratio transformation, one can reduce the backward equation satisfied by the expected first-passage time to a constant level for the mixed process to a differential-difference equation with a constant retardation. We discuss a method for solving this equation and present some numerical results on its solution. The accuracy of some approximations which are easier to calculate is investigated.

We consider an important class of integer valued stochastic processes characterised by a certain form of time dependent probability generating function (P.G.F.). Certain results are obtained concerning the correlation structure of such processes, and it is shown that these are considerably simplified in the case of a stationary distribution having been attained. The results are illustrated in the latter case using some commonly occurring processes belonging to the class under consideration.

Using a result of Bikjalis (1971) concerning the rate of convergence in the multidimensional central limit theorem we obtain informations about some limit distributions in multivariate renewal theory.

The vertices of the kth power of a directed path with n vertices are exposed one by one to a selector in some random order. At any time the selector can see the graph induced by the vertices that have already appeared. The selector's aim is to choose online the maximal vertex (i.e. the vertex with no outgoing edges). We give upper and lower bounds for the asymptotic behaviour of pn,kn1/(k+1), where pn,k is the probability of success under the optimal algorithm. In order to derive the upper bound, we consider a model in which the selector obtains some extra information about the edges that have already appeared. We give the exact asymptotics of the probability of success under the optimal algorithm in this case. In order to derive the lower bound, we analyse a site percolation process on a sequence of the kth powers of a directed path with n vertices.

When passing from two to more dimensions, the study of non-isotropic scattering transport processes, and chain molecules, which are both covered by the same mathematical model, becomes affected by the non-commutativity of rotations. The techniques developed in [2], together with results on radially symmetric direction changes, are used in this paper to obtain a functional central limit theorem for those random processes, with a suitable normalization, the limit being a Brownian motion process which is completely identified.

When a complex system such as the Apollo vehicle arrives on the Moon, certain parts may no longer function. To restore the system to a working state, these parts must be replaced by new ones; but the vehicle can carry only a limited number of spare parts and must also be able to leave at a specified time after arrival. This paper investigates an abstract (and simplified) version of the problem of the optimal maintenance procedure for such a system. As a bonus of the abstract formulation, the results obtained are applicable to a variety of specific systems.

We derive explicit closed expressions for the moment generating functions of whole collections of quantities associated with the waiting time till the occurrence of composite events in either discrete or continuous-time models. The discrete-time models are independent, or Markov-dependent, binary trials and the events of interest are collections of successes with the property that each two consecutive successes are separated by no more than a fixed number of failures. The continuous-time models are renewal processes and the relevant events are clusters of points. We provide a unifying technology for treating both the discrete and continuous-time cases. This is based on first embedding the problems into similar ones for suitably selected Markov chains or Markov renewal processes, and second, applying tools from the exponential family technology.

Let F be the infinite-horizon minimal cost function, and FKS the minimal cost function for horizon s and terminal cost function K. In Section 2 we define the ‘stable domain' (the set of K for which ), determine some of its properties, and relate these to the questions of stability of the process (whether ) and whether a given solution of the dynamic programming equation can be identified with F These ideas are developed in Sections 3 and 5 for various strengthenings and weakenings of the hypothesis of non-negative costs. In Section 5 we derive a new sufficient condition for stability and for characterisation of F.