The long-run average cost per unit time of operating a finite dam controlled by a policy (Lam Yeh (1985)) is determined when the cumulative input process is the integral of a Markov chain. A penalty cost which accrues continuously at a rate g(X(t)), where g is a bounded measurable function of the content, is also introduced. An example where the input rate is a two-state Markov chain is considered in detail to illustrate the computations.

In this paper we consider an explicit solution of an optimal stopping problem arising in connection with a dice game. An optimal stopping rule and the maximum expected reward in this problem can easily be computed by means of the distributions involved and the specific rules of the game

Criteria are determined for the variance to mean ratio to be greater than one (over-dispersed) or less than one (under-dispersed). This is done for random variables which are functions of a Markov chain in continuous time, and for the counts in a simple point process on the line. The criteria for the Markov chain are in terms of the infinitesimal generator and those for the point process in terms of the conditional intensity. Examples include a conjecture of Faddy (1994). The case of time-reversible point processes is particularly interesting, and here underdispersion is not possible. In particular, point processes which arise from Markov chains which are time-reversible, have finitely many states and are irreducible are always overdispersed.

Estimation of parameters in diffusion models is usually handled by maximum likelihood and involves the calculation of a Radon–Nikodym derivative. This methodology is often not available when minor changes are made to the model. However, these complications can usually be avoided and results obtained under more general conditions using quasi-likelihood methods. The basic ideas are explained in this paper and are illustrated through discussion of the Cox–Ingersoll–Ross model and a modification of the Langevin model.

We consider positive zero-sum stochastic games with countable state and action spaces. For each player, we provide a characterization of those strategies that are optimal in every subgame. These characterizations are used to prove two simplification results. We show that if player 2 has an optimal strategy then he/she also has a stationary optimal strategy, and prove the same for player 1 under the assumption that the state space and player 2's action space are finite.

In cell-size-dependent growth the probabilistic rate of division of a cell into daughter-cells and the rate of increase of its size depend on its size. In this paper the expected number of cells in the population at time t is calculated for a variety of models, and it is shown that population growths slower and faster than exponential are both possible. When the cell sizes are bounded conditions are given for exponential growth.

We introduce a large and flexible class of discrete tempered stable distributions, and analyze the domains of attraction for both this class and the related class of positive tempered stable distributions. Our results suggest that these are natural models for sums of independent and identically distributed random variables with tempered heavy tails, i.e. tails that appear to be heavy up to a point, but ultimately decay faster.

Customers arrive sequentially at times x1 < x2 < · · · < xn and stay for independent random times Z1, …, Zn > 0. The Z-variables all have the same distribution Q. We are interested in situations where the data are incomplete in the sense that only the order statistics associated with the departure times xi + Zi are known, or that the only available information is the order in which the customers arrive and depart. In the former case we explore possibilities for the reconstruction of the correct matching of arrival and departure times. In the latter case we propose a test for exponentiality.

For a random walk with both downward and upward jumps (increments), the joint distribution of the exit time across a given level and the undershoot or overshoot at crossing is determined through its generating function, when assuming that the distribution of the jump in the direction making the exit possible has a Laplace transform which is a rational function. The expected exit time is also determined and the paper concludes with exact distribution results concerning exits from bounded intervals. The proofs use simple martingale techniques together with some classical expansions of polynomials and Rouché's theorem from complex function theory.

In this article we consider a generalisation of conservative processes in which the usual transition rate parameters are replaced by time-dependent stochastic variables. The main result of the article shows that these generalised processes which we call conservative processes with stochastic rates have transition probabilities which can be characterised in terms of exchangeable random variables in a manner similar to the characterisation of conservative processes in terms of independent random variables given by Bartlett (1949). We use this characterisation to obtain general expressions for the transition probabilities and to examine some limiting aspects of the processes. The carrier-borne epidemic is treated as a particular case of these generalised processes.

A non-analytical proof of recurrence is obtained via an embedding procedure for a two-dimensional correlated lattice random walk.

The Poisson process is regarded as a point process of rare events because of the classical result that the number of successes in a sequence of Bernoulli trials is asymptotically Poisson as the probability of a success tends to 0. It is shown that this rareness property of the Poisson process is characteristic of any infinitely divisible point process or random measure with independent increments. These processes and measures arise as limits of certain rarefactions of compound point processes: purely atomic random measures with uniformly null atom sizes. Examples include thinnings and partitions of point processes.

In this paper a problem arising in queueing and dam theory is studied. We shall consider a G/G*/1 queueing model, i.e., a G/G/1 queueing model of which the service process is a separable centered process with stationary independent increments. This is a generalisation of the well-known G/G/1 model with constant service rate.

Several results concerning the amount of work done by the server, the busy cycles etc., are derived, mainly using the well-known method of Pollaczek. Emphasis is laid on the similarities and dissimilarities between the results of the ‘classical’ G/G/1 model and the G/G*/1 model.

We observe a sequence {Xk}k≧1 of i.i.d. non-negative random variables one at a time. After each observation, we select or reject the observed variable. A variable that is rejected may not be recalled. We want to select N variables as soon as possible subject to the constraint that the sum of the N selected variables does not exceed some prescribed value C > 0. In this paper, we develop a sequential selection procedure that minimizes the expected number of observed variables, and we study some of its properties. We also consider the situation where N → ∞and C/N → α > 0. Some applications are briefly discussed.

In this paper, central limit theorems for multivariate semi-Markov sequences and processes are obtained, both as the number of jumps of the associated Markov chain tends to infinity and, if appropriate, as the time for which the process has been running tends to infinity. The theorems are widely applicable since many functions defined on Markov or semi-Markov processes can be analysed by exploiting appropriate embedded multivariate semi-Markov sequences. An application to a problem in ion channel modelling is described in detail. Other applications, including to multivariate stationary reward processes, counting processes associated with Markov renewal processes, the interpretation of Markov chain Monte Carlo runs and statistical inference on semi-Markov models are briefly outlined.

This paper is concerned with the estimation of the variance for the multitype Galton-Watson process X = {Xn = (Xn(1),…, Xn(p)); n ≧ 0}. Two estimators for the variance matrix are obtained and asymptotic results for the estimators are given. The first is a maximum likelihood estimator based upon knowledge of individual offspring sizes, the second estimator is based on parent-offspring type combination counts only. Estimators for the asymptotic variances of the Asmussen and Keiding estimator and Becker estimator are also proposed.

Consider a population of distinct species Sj, j∈J, members of which are selected at different time points T1, T2,· ··, one at each time. Assume linear costs per unit of time and that a reward is earned at each discovery epoch of a new species. We treat the problem of finding a selection rule which maximizes the expected payoff. As the times between successive selections are supposed to be continuous random variables, we are dealing with a continuous-time optimal stopping problem which is the natural generalization of the one Rasmussen and Starr (1979) have investigated; namely, the corresponding problem with fixed times between successive selections. However, in contrast to their discrete-time setting the derivation of an optimal strategy appears to be much harder in our model as generally we are no longer in the monotone case.

This note gives a general point process formulation for this problem, leading in particular to an equivalent stopping problem via stochastic intensities which is easier to handle. Then we present a formal derivation of the optimal stopping time under the stronger assumption of i.i.d. (X1, A1) (X2, A2), · ·· where Xn gives the label (j for Sj) of the species selected at Tn and An denotes the time between the nth and (n – 1)th selection, i.e. An = Tn – Tn–1. In the case where even Xn and An are independent and An has an IFR (increasing failure rate) distribution, an explicit solution for the optimal strategy is derived as a simple consequence.

We investigate the asymptotic sample path behaviour of a randomly perturbed discrete-time dynamical system. We consider the case where the trajectories of the non-perturbed dynamical system are attracted by a finite number of limit sets and characterize a case where this property remains valid for the perturbed dynamical system when the perturbation converges to zero. For this purpose, no further assumptions on the perturbation are needed and our main condition applies to the limit sets of the non-perturbed dynamical system. When the limit sets reduce to limit points we show that this main condition is more general than the usual assumption of the existence of a Lyapunov function for the non-perturbed dynamical system. An application to an epidemic model is given to illustrate our results.

We extend Goldie's implicit renewal theorem to the arithmetic case, which allows us to determine the tail behavior of the solution of various random fixed point equations. It turns out that the arithmetic and nonarithmetic cases are very different. Under appropriate conditions we obtain that the tail of the solution X of the fixed point equations X =DAX + B and X =DAX ∨ B is ℓ(x) q(x) x-κ, where q is a logarithmically periodic function q(x eh) = q(x), x > 0, with h being the span of the arithmetic distribution of log A, and ℓ is a slowly varying function. In particular, the tail is not necessarily regularly varying. We use the renewal theoretic approach developed by Grincevičius (1975) and Goldie (1991).