We define a new family of special functions that we call lattice Bessel functions. They are indexed by the N-dimensional integer lattice such that they reduce to modified Bessel functions when N = 1, and the exponential function when N = 0. The transition probabilities for an M/M/1 queue going from one state to another before becoming idle (exiting at 0) can be solved in terms of modified Bessel functions. In this paper, we use lattice Bessel functions to solve the analogous problem involving the exit time from the interior of the positive orthant of the N-dimensional lattice for a series Jackson network with N nodes. These special functions allow us to derive asymptotic expansions for the taboo transition probabilities, as well as for the tail of the exit-time distribution.

Let N(t) be a birth-death process on 𝒩= {0, 1, 2, ·· ·} governed by the transition rates λn > 0 (n ≧ 0) and μn > 0 (n ≧ 1) where λn → λ > 0 and μn → μ > 0 as n → ∞ and ρ = λ/μ. Let Tmn be the first-passage time of N(t) from m to n and define It is shown that, when converges in distribution to TBP(μ,λ) as n → ∞ where TΒΡ (μ,λ) is the server busy period of an M/M/1 queueing system with arrival rate μ and service rate λ. Correspondingly T0n/E[T0n] converges to 1 with probability 1 as n →∞. Of related interest is the conditional first-passage time mTrn of N(t) from r to n given no visit to m where m < r < n. As we shall see, the conditional first-passage time of N(t) can be viewed as an ordinary first-passage time of a modified birth-death process M(t) governed by where are generated from λn and μn. Furthermore it is shown that for and while for and This enables one to establish the relation between the limiting behavior of the ordinary first-passage times and that of the conditional first-passage times.

A stochastic death process in which particles of two types combine and are removed from the system at a rate proportional to the product of their numbers is derived and analyzed. Exact expressions for the probability distribution and moments are given and an approximating diffusion is obtained. Numerical calculations show close agreement between the exact solution and this diffusion even for small numbers of particles, and reveal the relatively low level of stochastic variation.

It is shown that an elementary pure birth process is a mixed Poisson process iff the sequence of post-jump intensities forms a martingale with respect to the σ -fields generated by the jump times of the process. In this case, the post-jump intensities converge almost surely to the mixing random variable of the process.

Using a well-known transformation that preserves Wiener measure a theorem is proved that enables one to derive in a non-computational manner distributions of simple functionals of the so-called adjusted range process.

We consider the random vector T = (T(0), ···, T(n)) with independent identically distributed coordinates such that Pr{T(i) = j} = Pj, j = 0, 1, ···, n, Σ . A realization of T can be viewed as a random graph GT with vertices {0, ···, n} and arcs {(0, T(0)), ···, (n, T(n))}. For each T we partition the vertex-set of GT into three disjoint groups and study the joint probability distribution of their cardinalities. Assuming that we observe the asymptotics of this distribution, as n → ∞, for all possible values of P0. It turns out that in some cases these cardinalities are asymptotically independent and identically distributed.

Classes of distributions are defined in terms of antistarshapedness or dispersiveness. Necessary and sufficient conditions are given for these classes to be closed under mixtures. These conditions characterize distributions with log-concave densities.

This study is concerned with finite Markov decision processes whose dynamics and reward structure are unknown but the state is observable exactly.

We establish a learning algorithm which yields an optimal policy and construct an adaptive policy which is optimal under the average expected reward criterion.

Continuous random search methods with an average complexity given by O(log(1/ε)) for ε → 0 where ε is a given accuracy were presented in a recent paper. In this article an example of an O(log log(1/ε)) method is presented and illustrated.

A system is subject to a sequence of shocks occurring randomly at times n = 1, 2, ···; each shock causes a random amount of damage. The system might fail at any point in time n, and the probability of a failure depends on the history of the system. Upon failure the system is replaced by a new and identical system and a cost is incurred. If the system is replaced before failure a smaller cost is incurred. We study the problem of specifying a replacement rule which minimizes the long-run (expected) average cost per unit time. A special case, in which the system fails when the total damage first exceeds a fixed threshold, is analysed in detail.