A second-order random walk on a graph or network is a random walk where transition probabilities depend not only on the present node but also on the previous one. A notable example is the non-backtracking random walk, where the walker is not allowed to revisit a node in one step. Second-order random walks can model physical diffusion phenomena in a more realistic way than traditional random walks and have been very successfully used in various network mining and machine learning settings. However, numerous questions are still open for this type of stochastic processes. In this work, we extend well-known results concerning mean hitting and return times of standard random walks to the second-order case. In particular, we provide simple formulas that allow us to compute these numbers by solving suitable systems of linear equations. Moreover, by introducing the ‘pullback’ first-order stochastic process of a second-order random walk, we provide second-order versions of the renowned Kac’s and Random Target Lemmas.

The coefficient of tail dependence is a quantity that measures how extreme events in one component of a bivariate distribution depend on extreme events in the other component. It is well known that the Gaussian copula has zero tail dependence, a shortcoming for its application in credit risk modeling and quantitative risk management in general. We show that this property is shared by the joint distributions of hitting times of bivariate (uniformly elliptic) diffusion processes.

We describe in detail the speed of `coming down from infinity' for birth-and-death processes which eventually become extinct. Under general assumptions on the birth-and-death rates, we firstly determine the behavior of the successive hitting times of large integers. We identify two different regimes depending on whether the mean time for the process to go from n+1 to n is negligible or not compared to the mean time to reach n from ∞. In the first regime, the coming down from infinity is very fast and the convergence is weak. In the second regime, the coming down from infinity is gradual and a law of large numbers and a central limit theorem for the hitting times sequence hold. By an inversion procedure, we deduce that the process is almost surely equivalent to a nonincreasing function when the time goes to 0. Our results are illustrated by several examples including applications to population dynamics and population genetics. The particular case where the death rate varies regularly is studied in detail.

This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space.

The additive-increase multiplicative-decrease (AIMD) schemes designed to control congestion in communication networks are investigated from a probabilistic point of view. Functional limit theorems for a general class of Markov processes that describe these algorithms are obtained. The asymptotic behaviour of the corresponding invariant measures is described in terms of the limiting Markov processes. For some special important cases, including TCP congestion avoidance, an important autoregressive property is proved. As a consequence, the explicit expression of the related invariant probabilities is derived. The transient behaviour of these algorithms is also analysed.

For a compound Poisson dam with exponential jumps and linear release rate (shot-noise process), we compute the Laplace-Stieltjes transform (LST) and the mean of the hitting time of some positive level given that the process starts from some given positive level. The solution for the LST is in terms of confluent hypergeometric functions of the first and second kinds (Kummer functions).

Equipping the edges of a finite rooted tree with independent resistances that are inverse Gaussian for interior edges and reciprocal inverse Gaussian for terminal edges makes it possible, for suitable constellations of the parameters, to show that the total resistance is reciprocal inverse Gaussian (Barndorff-Nielsen 1994). This result is extended to infinite trees. Also, a connection to Brownian diffusion is established and, for the case of finite trees, an exact distributional and independence result is derived for the conditional model given the total resistance.

The Stein–Chen method for Poisson approximation is adapted to the setting of the geometric distribution. This yields a convenient method for assessing the accuracy of the geometric approximation to the distribution of the number of failures preceding the first success in dependent trials. The results are applied to approximating waiting time distributions for patterns in coin tossing, and to approximating the distribution of the time when a stationary Markov chain first visits a rare set of states. The error bounds obtained are sharper than those obtainable using related Poisson approximations.

The paper deals with processes in (or ), one of whose components is skip-free. We obtain identities for distributions of hitting times for the components of the process generalizing the well-known one for the one-dimensional case. These relations reflect the fact that in this case spatial and time coordinates play, in some sense, symmetric roles. They turn out to be useful for solving several problems. For example, they allow us to find the distribution of the number of jumps of the process, which fall in a fixed set before the skip-free component of the process hits a fixed level. Examples are given showing how our results can be applied to models in branching processes, queueing, and risk theory.

In this paper, bidimensional stochastic processes defined by ax(t) = y(t)dt and dy(t) = m(y)dt + [2v(y)]1/2dW(t), where W(t) is a standard Brownian motion, are considered. In the first section, results are obtained that allow us to characterize the moment-generating function of first-passage times for processes of this type. In Sections 2 and 5, functions are computed, first by fixing the values of the infinitesimal parameters m(y) and v(y) then by the boundary of the stopping region.

If X is a Brownian motion with drift and γ = inf{t > 0: Mt = t} we derive the joint density of the triple {U, γ, Δ}, where and Δ= γ —Xγ. In the case δ = 0 it follows easily from this that Δ has an Exp(2) distribution and this in turn implies the rather surprising result that if τ= inf{t > 0: Xt = Mt = t}, then Pr{τ = 0} = 0 and . We also derive various other distributional results involving the pair (X, M), including for example the distribution of ; in particular we show that, in case δ. = 1, when Pr{0 < τ < ∞} = 1, the ratio τ+/τ has the arc-sine distribution.

A direct approach to derive dependence properties among the hitting times of bivariate processes has been initiated by Ebrahimi (1987) and explored further by Ebrahimi and Ramallingam (1988). In this paper, new results are obtained for multivariate processes, which help us to identify positive and negative dependence structures among the hitting times of the processes. Applications of our theorems to reliability of systems are given.

An exponential martingale is defined for a class of random walks in the positive quarter lattice which are associated with a wide variety of Markovian two-queue networks. Balance formulas generalizing Wald's exponential identity are derived from the regularity of several types of hitting times with respect to this martingale. In a queuing context, these formulas can be interpreted as functional relations of practical interest between the number of customers at certain epochs and the utilization of the queues up to these epochs.

For regular Markov processes on a countable space, we provide criteria for the forms of ergodicity in the title in terms of the existence of solutions to inequalities involving the Q-matrix of the process. An application to birth-death processes is given.

For finite Markov chains, the concepts of ergodicity and strong ergodicity are equivalent, but this is not necessarily the case when the state space is infinite. In this note we give some new characterizations of strong ergodicity. These lead to simple necessary or sufficient criteria for strong ergodicity, which readily enable us to classify a number of examples.

Suppose a physical process is modelled by a Markov chain with transition probability on S1 ∪ S2, S1 denoting the transient states and S2 a set of absorbing states.

If v denotes the output distribution on S2, the question arises as to what input distributions (of raw materials) on S1 produce v. In this note we give an alternative to the formulation of Ray and Margo [2] and reduce the problem to one system of linear inequalities. An application to random walk is given and the equiprobability case examined in detail.