Recent results on the winding angle of the ordinary two-dimensional random walk on the integer lattice are reviewed. The difference between the Brownian motion winding angle and the random walk winding angle is discussed. Other functionals of the random walk, such as the maximum winding angle, are also considered and new results on their asymptotic behavior, as the number of steps increases, are presented. Results of computer simulations are presented, indicating how well the asymptotic distributions fit the exact distributions for random walks with 10m steps, for m = 2, 3, 4, 5, 6, 7.

Invariant stochastic properties are studied for circuit processes, i.e. processes whose sample paths can be represented by a denumerable set of overlapping directed circuits c in a denumerable set S and a set of positive weights , when the family of the weights varies.

The points of a graph G will form clusters as a result of a flow process. Initially, points i of G own resources xi which are i.i.d. random real numbers. Afterwards, resources flow between points, but always from a point to a neighbor that has accumulated a larger total resource. Thus points with small resource tend to lose it and points with large resource tend to gain. Eventually the flow stops with only two kinds of points, nulls with no resource left and absorbers with such large resource that no neighbor can take it. The final resource at an absorber is a sum of certain initial resources xi, and the corresponding points i form one cluster. Analytical results are obtainable when G is the chain of integer points on the line. Probability distributions are derived for the distance between consecutive absorbers and the size of a cluster. Indeed these distributions do not involve the given distribution for the xi. The Laplace transform of the distribution of final resources at absorbers is derived but the distribution itself is obtained by a simulation. For general graphs G only partial results are obtained.

Sufficient conditions are given under which two simple point processes on the positive half-line can be stochastically compared as random elements of D(0,∞) or R∞+ Using a martingale approach to point processes, the conditions are proposed via a compensator function family. Appropriate versions of the processes being compared are constructed on the same probability space. The results are illustrated by replacement policies and semi-Markov point processes.

This paper extends the rate conservation principle to cadlag processes whose jumps form a non-stationary point process with a time-dependent intensity. It is shown that this is a direct consequence of path integration and the strong law of large numbers for local martingales. When specialized to mean rates a non-stationary version of Miyazawa's result is obtained which is recovered in the stationary case. Some applications of the result are also given.

We study a search model in which offers of random size are received randomly over time. The arrival times form a point process of a certain type, and the offer size distribution may depend on the corresponding arrival time. The search costs have a time-dependent cost rate. The objective is to stop the search process such that the expected discounted net reward (associated with the maximal offer received so far) is maximized. A stopping time σ is suggested, and conditions are specified under which σ turns out to be optimal.

We consider the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. We use a stationary randomized control scheme for forcing and compute a maximum likelihood estimate of the unknown parameter from the resulting observations. We obtain an exponential upper bound on the rate of decay of the probability of error. This allows us to choose the rate of forcing appropriately, whereby we achieve a o(f(n) log n) learning loss for any function as .

Suppose that π is a policy for resource allocation in a stochastic environment and π ∗ is an optimal policy. Two existing procedures for policy evaluation are described and compared. Both of these evaluate π by means of upper bounds on R(π ∗) – R(π), the total reward lost when making resource allocations according to π rather than π∗. The bounds developed by these two methods are called Type 1 and Type 2. We demonstrate by example that neither of these procedures dominates the other in the sense of always yielding tighter bounds. A modification to Type 2 bounds is proposed resulting in an improved procedure which always dominates the Type 1 approach.

The classical proofs for the existence of a stationary (s, S) inventory policy that minimizes the total discounted or average cost over an infinite horizon are lengthy because they depend heavily on the optimality results for corresponding finite-horizon models. This note presents a simpler alternative. Since optimal stationary (s, S) policies are relatively simple to characterize, it is easy to construct a solution to the optimality equation which is satisfied by an (s, S) policy or an equivalent variant thereof. For the discounted model, the proof characterizes an (s, S) policy that is optimal for all initial inventory positions. This policy can be generated by a simple existing algorithm. For the average-cost model, the optimality proof is completed with some additional arguments, which are simple but novel, to overcome the normal difficulties encountered in models with unbounded one-step expected costs.

Analogues of some theorems of Brown (1980) concerning renewal measures with DFR or IMRL underlying distributions are proved for delayed renewal measures. Some related results, such as partial converses of the main theorems, are also presented.

In this note, an irreducible semi-Markov process is considered whose finite state space is partitioned into two non-empty sets A and B. Let MB(t) stand for the number of visits of Y to B during the time interval [0, t], t > 0. A renewal argument is used to derive closed-form expressions for the Laplace transform (with respect to t) of a certain family of functions in terms of which the moments of MB(t) are easily expressible. The theory is applied to a small reliability model in conjunction with a Tauberian argument to evaluate the behaviour of the first two moments of MB(t) as t →∞.

In this study an asymptotical analysis of the reliability of a complex renewable system with an unbounded number of repair units is provided. The system state is given by a binary vector e(t) = [e1(t), · ··, en(t)], ei(t) = 0(1), if at moment t the ith element is failure-free (failed). We assume that at the state e the ith element has failure intensity λi(e). At the instant of failure of every element the renewal work begins and the renewal time has distribution function Gi(t). Let E_ be the set of failed system states. The goal of this study is the asymptotic estimation of the distribution of the time until the first system failure, .

In this paper, we study a repair replacement model for a stochastically deteriorating system. For the expected discounted reward case, we show that the optimal replacement policy is of the form ‘replace at the time of the Nth failure'.

We consider the asymmetric simple exclusion process which starts from a product measure such that all the sites to the left of zero (including zero) are occupied and the right of 0 (excluding 0) are empty. We label the particle initially at 0 as the leading particle. We study the long-term behaviour of this process near large sites when the leading particle's holding time is different from that of the other particles. In particular, we assume that the leading particle moves at a slower rate than the other particles. We call this modified asymmetric simple exclusion process the road-hog process. Coupling and stochastic ordering techniques are used to derive the density profile of this process. Road-hog processes are useful in modelling series of exponential queues with Poisson and non-Poisson input process. The density profiles dramatically illustrate the flow of customers through the queues.

Let Tx be the length of time to accumulate x units of a resource. In queueing, the resource could be service. We derive a sufficient condition for the process to have stationary increments where Tx is an additive functional of a Markov process. This condition is satisfied in symmetric queues and generalized semi-Markov schemes with insensitive components. As a corollary, we show that the conditional expected response time in a symmetric queue is linear in the service requirement. A similar result holds for the conditional average residence time of an insensitive component in a GSMS.

In this paper we find the waiting time distribution in the transient domain and the busy period distribution of the GI G/1 queue. We formulate the problem as a two-dimensional Lindley process and then transform it to a Hilbert factorization problem. We achieve the solution of the factorization problem for the GI/R/1, R/G/1 queues, where R is the class of distributions with rational Laplace transforms. We obtain simple closed-form expressions for the Laplace transforms of the waiting time distribution and the busy period distribution. Furthermore, we find closed-form formulae for the first two moments of the distributions involved.

Given a realisation of a Markov chain, one can count the numbers of state transitions of each type. One can ask how many realisations are there with these transition counts and the same initial state. Whittle (1955) has answered this question, by finding an explicit though complicated formula, and has also shown that each realisation is equally likely. In the analysis of DNA sequences which comprise letters from the set {A, C, G, T}, it is often useful to count the frequency of a pattern, say ACGCT, in a long sequence and compare this with the expected frequency for all sequences having the same start letter and the same transition counts (or ‘dinucleotide counts' as they are called in the molecular biology literature). To date, no exact method exists; this paper rectifies that deficiency.

Let (Zn) be a p-type positively regular and non-singular critical Galton–Watson process with finite second moments. Associated with the spectral radius 1 of the mean matrix of (Zn) consider the right eigenvector u = (u1, · ··, up) > 0, and set . It is shown that lim inf, lim sup whenever Z0 = i, where .

The centred, periodic, stationary Gaussian process X(z), ≧ z ≧ 1 with covariances , appears when one studies the solutions of the vibrating string equation forced by noise, corresponding to the case of a finite string with the extremes tied together. The close relationship between this process and a Brownian bridge permits us to compute the distribution of the maximum excursion of the string at particular times.