Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation measure of a Markov chain. In particular, we establish weak convergence of the state occupation measure under a scaling of the generator matrix. Then, relying on the connection between the state occupation measure and the Dynkin martingale, we provide sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure. We apply our results to derive diffusion limits for the Markov-modulated Erlang loss model and the regime-switching Cox–Ingersoll–Ross process.

In this paper, we solve exit problems for a one-sided Markov additive process (MAP) which is exponentially killed with a bivariate killing intensity $\omega(\cdot,\cdot)$ dependent on the present level of the process and the current state of the environment. Moreover, we analyze the respective resolvents. All identities are expressed in terms of new generalizations of classical scale matrices for MAPs. We also remark on a number of applications of the obtained identities to (controlled) insurance risk processes. In particular, we show that our results can be applied to the Omega model, where bankruptcy takes place at rate $\omega(\cdot,\cdot)$ when the surplus process becomes negative. Finally, we consider Markov-modulated Brownian motion (MMBM) as a special case and present analytical and numerical results for a particular choice of piecewise intensity function $\omega(\cdot,\cdot)$ .

In this paper we consider an Ornstein–Uhlenbeck (OU) process (M(t))t≥0 whose parameters are determined by an external Markov process (X(t))t≥0 on a finite state space {1, . . ., d}; this process is usually referred to as Markov-modulated Ornstein–Uhlenbeck. We use stochastic integration theory to determine explicit expressions for the mean and variance of M(t). Then we establish a system of partial differential equations (PDEs) for the Laplace transform of M(t) and the state X(t) of the background process, jointly for time epochs t = t1, . . ., tK. Then we use this PDE to set up a recursion that yields all moments of M(t) and its stationary counterpart; we also find an expression for the covariance between M(t) and M(t + u). We then establish a functional central limit theorem for M(t) for the situation that certain parameters of the underlying OU processes are scaled, in combination with the modulating Markov process being accelerated; interestingly, specific scalings lead to drastically different limiting processes. We conclude the paper by considering the situation of a single Markov process modulating multiple OU processes.

Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and nondefective processes, and all possible scenarios, we identify the corresponding potential measures, which help to generalize a number of results for one-sided Lévy processes. The resulting rather neat formulae have various applications in risk and queueing theories, and, in particular, they lead to quasistationary distributions of the corresponding processes.

Let {Zn}n≥0 be a random walk with a negative drift and independent and identically distributed increments with heavy-tailed distribution, and let M = supn≥0Zn be its supremum. Asmussen and Klüppelberg (1996) considered the behavior of the random walk given that M > x for large x, and obtained a limit theorem, as x → ∞, for the distribution of the quadruple that includes the time τ = τ(x) to exceed level x, position Zτ at this time, position Zτ-1 at the prior time, and the trajectory up to it (similar results were obtained for the Cramér-Lundberg insurance risk process). We obtain here several extensions of this result to various regenerative-type models and, in particular, to the case of a random walk with dependent increments. Particular attention is given to describing the limiting conditional behavior of τ. The class of models includes Markov-modulated models as particular cases. We also study fluid models, the Björk-Grandell risk process, give examples where the order of τ is genuinely different from the random walk case, and discuss which growth rates are possible. Our proofs are purely probabilistic and are based on results and ideas from Asmussen, Schmidli and Schmidt (1999), Foss and Zachary (2002), and Foss, Konstantopoulos and Zachary (2007).

We study a network of fluid queues in which exogenous arrivals are modulated by a continuous-time Markov chain. Service rates in each queue are proportional to the queue size, and the network is assumed to be irreducible. The queue levels satisfy a linear, vector-valued differential equation. We obtain joint moments of the queue sizes recursively, and deduce the Laplace transform of the queue sizes in the stationary regime.

We consider growth-collapse processes (GCPs) that grow linearly between random partial collapse times, at which they jump down according to some distribution depending on their current level. The jump occurrences are governed by a state-dependent rate function r(x). We deal with the stationary distribution of such a GCP, (Xt)t≥0, and the distributions of the hitting times Ta = inf{t ≥ 0 : Xt = a}, a > 0. After presenting the general theory of these GCPs, several important special cases are studied. We also take a brief look at the Markov-modulated case. In particular, we present a method of computing the distribution of min[Ta, σ] in this case (where σ is the time of the first jump), and apply it to determine the long-run average cost of running a certain Markov-modulated disaster-ridden system.

We establish new multidimensional martingales for Markov additive processes and certain modifications of such processes (e.g., such processes with reflecting barriers). These results generalize corresponding one-dimensional martingale results for Lévy processes. This martingale is then applied to various storage processes, queues and Brownian motion models.

The risk reserve process of an insurance company within a deteriorating Markov-modulated environment is considered. The company invests its capital with interest rate α; the premiums and claims are increasing with rates β and γ. The problem of stopping the process at a random time which maximizes the expected net gain in order to calculate new premiums is investigated. A semimartingale representation of the risk reserve process yields, under certain conditions, an explicit solution of the problem.

We consider storage models where the input rate and the demand are modulated by a Markov jump process. One particular example from teletraffic theory is a fluid model of a multiplexer loaded by exponential on-off sources. Although the storage level process has been widely studied, little attention has been paid to the output rate process. We will show that, under certain assumptions, there exists another Markov jump process that modulates the output rate. The modulating process is explicitly constructed. It turns out to be a modification of a GI/G/1 queueing process