Let (Mn,Sn)n≥0 be a Markov random walk with positive recurrent driving chain (Mn)n≥0 on the countable state space 𝒮 with stationary distribution π. Suppose also that lim supn→∞Sn=∞ almost surely, so that the walk has almost-sure finite strictly ascending ladder epochs σn>. Recurrence properties of the ladder chain (Mσn>)n≥0 and a closely related excursion chain are studied. We give a necessary and sufficient condition for the recurrence of (Mσn>)n≥0 and further show that this chain is positive recurrent with stationary distribution π> and 𝔼π>σ1><∞ if and only if an associated Markov random walk (𝑀̂n,𝑆̂n)n≥0, obtained by time reversal and called the dual of (Mn,Sn)n≥0, is positive divergent, i.e. 𝑆̂n→∞ almost surely. Simple expressions for π> are also provided. Our arguments make use of coupling, Palm duality theory, and Wiener‒Hopf factorization for Markov random walks with discrete driving chain.

Let 𝓈 be a finite or countable set. Given a matrix F = (Fij)i,j∈𝓈 of distribution functions on R and a quasistochastic matrix Q = (qij)i,j∈𝓈, i.e. an irreducible nonnegative matrix with maximal eigenvalue 1 and associated unique (modulo scaling) positive left and right eigenvectors u and v, the matrix renewal measure ∑n≥0Qn ⊗ F*n associated with Q ⊗ F := (qijFij)i,j∈𝓈 (see below for precise definitions) and a related Markov renewal equation are studied. This was done earlier by de Saporta (2003) and Sgibnev (2006, 2010) by drawing on potential theory, matrix-analytic methods, and Wiener-Hopf techniques. In this paper we describe a probabilistic approach which is quite different and starts from the observation that Q ⊗ F becomes an ordinary semi-Markov matrix after a harmonic transform. This allows us to relate Q ⊗ F to a Markov random walk {(Mn, Sn)}n≥0 with discrete recurrent driving chain {Mn}n≥0. It is then shown that renewal theorems including a Choquet-Deny-type lemma may be easily established by resorting to standard renewal theory for ordinary random walks. The paper concludes with two typical examples.

Motivated by the study of the asymptotic normality of the least-squares estimator in the (autoregressive) AR(1) model under possibly infinite variance, in this paper we investigate a self-normalized central limit theorem for Markov random walks. That is, let {Xn, n ≥ 0} be a Markov chain on a general state space X with transition probability P and invariant measure π. Suppose that an additive component Sn takes values on the real line , and is adjoined to the chain such that {Sn, n ≥ 1} is a Markov random walk. Assume that Sn = ∑k=1nξk, and that {ξn, n ≥ 1} is a nondegenerate and stationary sequence under π that belongs to the domain of attraction of the normal law with zero mean and possibly infinite variance. By making use of an asymptotic variance formula of Sn / √n, we prove a self-normalized central limit theorem for Sn under some regularity conditions. An essential idea in our proof is to bound the covariance of the Markov random walk via a sequence of weight functions, which plays a crucial role in determining the moment condition and dependence structure of the Markov random walk. As illustrations, we apply our results to the finite-state Markov chain, the AR(1) model, and the linear state space model.

We prove a d-dimensional renewal theorem, with an estimate on the rate of convergence, for Markov random walks. This result is applied to a variety of boundary crossing problems for a Markov random walk (Xn,Sn), n ≥0, in which Xn takes values in a general state space and Sn takes values in ℝd. In particular, for the case d = 1, we use this result to derive an asymptotic formula for the variance of the first passage time when Sn exceeds a high threshold b, generalizing Smith's classical formula in the case of i.i.d. positive increments for Sn. For d > 1, we apply this result to derive an asymptotic expansion of the distribution of (XT,ST), where T = inf { n : Sn,1 > b } and Sn,1 denotes the first component of Sn.

Previous work in extending Wald's equations to Markov random walks involves finiteness of moment generating functions and uniform recurrence assumptions. By using a new approach, we can remove these assumptions. The results are applied to establish finiteness of moments of ladder variables and to derive asymptotic expansions for expected first passage times of Markov random walks. Wiener–Hopf factorizations for Markov random walks are also applied to analyse ladder variables and related first passage problems.

Let (X, S) = {(Xn, Sn); n ≧0} be a Markov random walk with finite state space. For a ≦ 0 < b define the stopping times τ= inf {n:Sn > b} and T= inf{n:Sn∉(a, b)}. The diffusion approximations of a one-barrier probability P {τ < ∝ | Xo= i}, and a two-barrier probability P{ST ≧b | Xo = i} with correction terms are derived. Furthermore, to approximate the above ruin probabilities, the limiting distributions of overshoot for a driftless Markov random walk are involved.