We examine necessary and sufficient conditions for recurrence and positive recurrence of a class of irreducible, level-dependent quasi-birth-and-death (LDQBD) processes with a block tridiagonal structure that exhibits asymptotic convergence in the rows as the level tends to infinity. These conditions are obtained by exploiting a multi-dimensional Lyapunov drift approach, along with the theory of generalized Markov group inverses. Additionally, we highlight analogies to well-known average drift results for level-independent quasi-birth-and-death (QBD) processes.

We study the conditions for positive recurrence and transience of multi-dimensional birth-and-death processes describing the evolution of a large class of stochastic systems, a typical example being the randomly varying number of flow-level transfers in a telecommunication wire-line or wireless network. First, using an associated deterministic dynamical system, we provide a generic method to construct a Lyapunov function when the drift is a smooth function on ℝN. This approach gives an elementary and direct proof of ergodicity. We also provide instability conditions. Our main contribution consists of showing how discontinuous drifts change the nature of the stability conditions and of providing generic sufficient stability conditions having a simple geometric interpretation. These conditions turn out to be necessary (outside a negligible set of the parameter space) for piecewise constant drifts in dimension two.

Juggler's exclusion process describes a system of particles on the positive integers where particles drift down to zero at unit speed. After a particle hits zero, it jumps into a randomly chosen unoccupied site. We model the system as a set-valued Markov process and show that the process is ergodic if the family of jump height distributions is uniformly integrable. In a special case where the particles jump according to a set-avoiding memoryless distribution, the process reaches its equilibrium in finite nonrandom time, and the equilibrium distribution can be represented as a Gibbs measure conforming to a linear gravitational potential.

This paper is devoted to studying an extended class of time-continuous branching processes, motivated by the study of stochastic control theory and interacting particle systems. The uniqueness, extinction, recurrence and positive recurrence criteria for the processes are presented. The main new point in our proofs is the use of several different comparison methods. The resulting picture shows that the methods are effective and hence should also be meaningful in other situations.

For a given transition rate, i.e., a Q-matrix Q = (qij) on a countable state space, the uniqueness of the Q-semigroup P(t) = (Pij(t)), the recurrence and the positive recurrence of the corresponding Markov chain are three fundamental and classical problems, treated in many textbooks. As an addition, this paper introduces some practical results motivated from the study of a type of interacting particle systems, reaction diffusion processes. The main results are theorems (1.11), (1.17) and (1.18). Their proofs are quite straightforward.

It is shown how Markov chain theory can be exploited to study non-linear time series, the emphasis being on the classification into stationary and non-stationary models. A generalized h-step version of the Tweedie (1975), (1976) criteria is formulated, and applications are given to a number of non-linear models. New results are obtained, and known results are shown to emerge as special cases in both the scalar and vector case. A connection to stability theory is briefly discussed, and it is indicated how the Markov property can be utilized for estimation purposes.

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Continuous time one-type branching processes allowing immigration are considered. The invariant measure, which is shown to be unique, is exhibited. From this, a condition for positive recurrence similar to that of Heathcote's in the discrete time case is obtained. For the critical discrete time case, Seneta's sufficient condition for positive recurrence is improved to give a necessary and sufficient condition.