The idea is developed of imbedding a given Markov process in a more general Markov process in a definite sense, the sense of weak coupling. It is shown that this gives a natural theory of insensitivity in that, the stronger the balance conditions satisfied by the imbedded process, the weaker the conditions that are required of the imbedding process. The imbeddings associated with a range of balance conditions are discussed.

A classical result in queueing theory states that in the stable M/G/1 queue, the stationary distribution W(x) of the waiting time of an arriving customer or of the virtual waiting time satisfies a linear Volterra integral equation of the second kind, of convolution type. For many variants of the M/G/1 queue, there are corresponding integral equations, which in most cases differ from the Pollaczek–Khinchin equation only in the form of the inhomogeneous term. This leads to interesting factorizations of the waiting-time distribution and to substantial algorithmic simplifications. In a number of priority queues, the waiting-time distributions satisfy Volterra integral equations whose kernel is a functional of the busy-period distribution in related M/G/1 queues. In other models, such as the M/G/1 queue with Bernoulli feedback or with limited admissions of customers per service, there is a more basic integral equation of Volterra type, which yields a probability distribution in terms of which the waiting-time distributions are conveniently expressed.

For several complex queueing models with an embedded Markov renewal process of M/G/1 type, one obtains matrix Volterra integral equations for the waiting-time distributions or for related vectors of mass functions. Such models include the M/SM/1 and the N/G/1 queues, as well as the M/G/1 queue with some forms of bulk service.

When the service-time distributions are of phase type, the numerical computation of waiting-time distributions may commonly be reduced to the solution of systems of linear differential equations with constant coefficients.

The spectral factorization problem was solved in Hallin (1984) for the class of (non-stationary) m-variate MA(q) stochastic processes, i.e. the class of second-order q-dependent processes. It was shown that such a process generally admits an infinite (mq(mq +1)/2-dimensional) family of possible MA(q) representations. The present paper deals with the invertibility properties and asymptotic behaviour of these MA(q) models, in connection with the problem of producing asymptotically efficient forecasts. Invertible and borderline non-invertible models are characterized (Theorems 3.1 and 3.2). A criterion is provided (Theorem 4.1) by which it can be checked whether a given MA model is a Wold–Cramér decomposition or not; and it is shown (Theorem 4.2) that, under mild conditions, almost every MA model is asymptotically identical with some Wold–Cramér decomposition. The forecasting problem is investigated in detail, and it is established that the relevant invertibility concept, with respect to asymptotic forecasting efficiency, is what we define as Granger-Andersen invertibility rather than the classical invertibility concept (Theorem 5.3). The properties of this new invertibility concept are studied and contrasted with those of its classical counterpart (Theorems 5.2 and 5.4). Numerical examples are also treated (Section 6), illustrating the fact that non-invertible models may provide asymptotically efficient forecasts, whereas invertible models, in some cases, may not. The mathematical tools throughout the paper are linear difference equations (Green's matrices, adjoint operators, dominated solutions, etc.), and a matrix generalization of continued fractions.

Given a d-dimensional random field and a Poisson process independent of it, suppose that it is possible to observe only the location of each point of the Poisson process and the value of the random field at that (randomly located) point. Non-parametric estimators of the mean and covariance function of the random field—based on observation over compact sets of single realizations of the Poisson samples—are constructed. Under fairly mild conditions these estimators are consistent (in various senses) as the set of observation becomes unbounded in a suitable manner. The state estimation problem of minimum mean-squared error reconstruction of unobserved values of the random field is also examined.

For a class of models of adaptive queueing systems an exact diffusion approximation is derived with the aim of obtaining information on the evolution of the systems. Our approximating diffusion process includes the Wiener and the Ornstein–Uhlenbeck processes with reflecting boundaries at 0. The goodness of the approximations is thoroughly discussed and the closed-form solutions obtained for the diffusion processes are compared with those holding for the queueing system in order to investigate the conditions under which reliable information can be obtained from the approximating continuous models. For the latter the transient behaviour is quantitatively analysed and the distribution of the busy period is determined in closed form.

The theory of allocation indices for defining the optimal policy in multi-armed bandit problems developed by Gittins is presented in the continuous-time case where the projects (or ‘arms’) are strong Markov processes. Complications peculiar to the continuous-time case are discussed. This motivates investigation of whether approximation of the continuous-time problems by discrete-time versions provides a valid technique with convergent allocation indices and optimal expected rewards. Conditions are presented under which the convergence holds.

In this note we deal with the stochastic difference equation of the form Yn+1 = AnYn + Bn, n∊ℤ, where the sequence is assumed to be strictly stationary and ergodic. By means of simple arguments a unique stationary solution of this equation is constructed. The stability of the stationary solution is the second subject of investigation. It is shown that under some additional assumptions

There are many examples of a device suffering damage from random environmental shocks. We model the damage level of such a device as a pure jump Markov process, where the incremental damage caused by a shock depends both on the magnitude of the shock and on the damage level just before the shock. We also look at the time until failure of the device, which occurs when the damage level exceeds a random threshold. The distribution of the failure time and the failure rate are examined, and conditions for the failure rate to be increasing or to have an increasing average are found.

Many queues and related stochastic models, and in particular those that have a matrix-geometric stationary probability vector, have steady-state queue-length densities that are asymptotically geometric. The graph of the asymptotic rate η of these densities as a function of the traffic intensity ρ is the caudal characteristic curve. This is an informative graph from which a number of qualitative inferences about the behavior of the queue may be drawn.

The caudal characteristic curve may be computed (by elementary algorithms) for several useful models for which a complete exact numerical solution is not practically feasible. These include queues with certain types of superimposed arrival processes and/or multiple non-exponential servers. The necessary theorems which lead to the algorithmic procedures as well as the interpretation of several numerical examples are discussed.

Simulated annealing is a randomized algorithm which has been proposed for finding globally optimum least-cost configurations in large NP-complete problems with cost functions which may have many local minima. A theoretical analysis of simulated annealing based on its precise model, a time-inhomogeneous Markov chain, is presented. An annealing schedule is given for which the Markov chain is strongly ergodic and the algorithm converges to a global optimum. The finite-time behavior of simulated annealing is also analyzed and a bound obtained on the departure of the probability distribution of the state at finite time from the optimum. This bound gives an estimate of the rate of convergence and insights into the conditions on the annealing schedule which gives optimum performance.

We analyse a stack protocol of the Capetanakis–Tsybakov–Mikhailov type for resolving collisions in a random multiple-access channel. We obtain a functional equation for the generating function of the expected collision resolution interval (CRI) durations, which is non-local with a non-commutative iteration semigroup. Using Mellin transform techniques and geometric properties of the iteration semigroup we show that for arrival rates smaller than a fixed threshold, the mean CRI duration for n initial colliders is asymptotically proportional to n. Ergodicity conditions are also demonstrated.

This paper considers a queueing system with two classes of customers and a single server, where the service policy is of threshold type. As soon as the amount of work required by the class 1 customers is greater than a fixed threshold, the class 1 customers get the server's attention; otherwise the class 2 customers have the priority. Service interruptions can occur for both classes of customers on the basis of the above description of the service mechanism, and in this case the service interruption discipline is preemptive resume priority (PRP). This model, which turns out to be a generalization of the PRP queueing system, has potential applications in computer systems and in communication networks. For Poisson inputs, exponential (arbitrary) servicetime distribution for class 1 (class 2) customers, we derive the Laplace–Stieltjes transform of the stationary joint distribution of the workload of the server, by reducing the analysis to the resolution of a boundary value problem. Explicit formulas are obtained.

Limit theorems of Berman involve the total time spent by Brownian motion with negative drift above a fixed or exponentially distributed negative level. We give explicitly the probability densities and distribution functions, obtained via an equivalence of laws.

This paper is concerned with blocking and loss probabilities in circuit-switched networks. We show that when the capacity of links and the offered traffic are increased together, a limiting regime emerges in which loss probabilities are as if links block independently, with blocking probabilities given by the solution of a simple convex programming problem. We then show that an approximate procedure, based on solving Erlang&s formula under the assumption of independent blocking, produces a unique solution when routes are fixed, and that under the limiting regime the estimates of loss probabilities obtained from the procedure converge to the correct values.

We examine a replacement model for a semi-Markov shock model with additive damage. Sufficient conditions are given for the optimality of control limit policies. The paper generalizes and unifies previous research in the area.

In addition, we investigate in detail the practical modelling and computational aspects of the replacement problem using a semi-Markov modelling structure.

This paper presents a new technique for analyzing the frequency of a very large class of rare events in large traffic systems. The method is based on the theory of large deviations. If n is a large parameter, typically the number of potential traffic sources, then where I is the solution to an associated variational problem. We present a new analysis of a previously solved system as well as an analysis of a previously intractable system. As by-products of our analysis, we obtain estimates of the transient behavior of the system, and show how they may be used in analyzing some flow control schemes.

This paper contains two contributions to the insensitivity theory for generalized semi-Markov schemes, namely

1. (1) a detailed account of a close relationship between insensitive schemes and partially balanced discrete-state Markov chains, and

2. (2) an extension of the notion of an insensitive scheme in order to incorporate insensitivity phenomena not previously covered.

Recently many authors have established connections between dispersive ordering and some other partial orderings of distributions. This paper presents the connection which superadditive ordering has with dispersive ordering.

Let X1, · ··, Xn be exchangeable random variables with finite variance and two sequences of constants satisfying a1≦···≦an, b1≦···≦bn. Suppose that a′1, ···, a′n is a rearrangement of a1, ···, an and that g(x) is a non-decreasing function. Then