Continuous-time Markov chains are frequently used to model the stochastic dynamics of (bio)chemical reaction networks. However, except in very special cases, they cannot be analyzed exactly. Additionally, simulation can be computationally intensive. An approach to address these challenges is to consider a more tractable diffusion approximation. Leite and Williams (Ann. Appl. Prob. 29, 2019) proposed a reflected diffusion as an approximation for (bio)chemical reaction networks, which they called the constrained Langevin approximation (CLA) as it extends the usual Langevin approximation beyond the first time some chemical species becomes zero in number. Further explanation and examples of the CLA can be found in Anderson et al. (SIAM Multiscale Modeling Simul. 17, 2019).

In this paper, we extend the approximation of Leite and Williams to (nearly) density-dependent Markov chains, as a first step to obtaining error estimates for the CLA when the diffusion state space is one-dimensional, and we provide a bound for the error in a strong approximation. We discuss some applications for chemical reaction networks and epidemic models, and illustrate these with examples. Our method of proof is designed to generalize to higher dimensions, provided there is a Lipschitz Skorokhod map defining the reflected diffusion process. The existence of such a Lipschitz map is an open problem in dimensions more than one.

By the technique of augmented truncations, we obtain the perturbation bounds on the distance of the finite-time state distributions of two continuous-time Markov chains (CTMCs) in a type of weaker norm than the V-norm. We derive the estimates for strongly and exponentially ergodic CTMCs. In particular, we apply these results to get the bounds for CTMCs satisfying Doeblin or stochastically monotone conditions. Some examples are presented to illustrate the limitation of the V-norm in perturbation analysis and to show the quality of the weak norm.

Birth–death processes form a natural class where ideas and results on large deviations can be tested. We derive a large-deviation principle under an assumption that the rate of jump down (death) grows asymptotically linearly with the population size, while the rate of jump up (birth) grows sublinearly. We establish a large-deviation principle under various forms of scaling of the underlying process and the corresponding normalization of the logarithm of the large-deviation probabilities. The results show interesting features of dependence of the rate functional upon the parameters of the process and the forms of scaling and normalization.

This paper investigates tail asymptotics of stationary distributions and quasi-stationary distributions (QSDs) of continuous-time Markov chains on subsets of the non-negative integers. Based on the so-called flux-balance equation, we establish identities for stationary measures and QSDs, which we use to derive tail asymptotics. In particular, for continuous-time Markov chains with asymptotic power law transition rates, tail asymptotics for stationary distributions and QSDs are classified into three types using three easily computable parameters: (i) super-exponential distributions, (ii) exponential-tailed distributions, and (iii) sub-exponential distributions. Our approach to establish tail asymptotics of stationary distributions is different from the classical semimartingale approach, and we do not impose ergodicity or moment bound conditions. In particular, the results also hold for explosive Markov chains, for which multiple stationary distributions may exist. Furthermore, our results on tail asymptotics of QSDs seem new. We apply our results to biochemical reaction networks, a general single-cell stochastic gene expression model, an extended class of branching processes, and stochastic population processes with bursty reproduction, none of which are birth–death processes. Our approach, together with the identities, easily extends to discrete-time Markov chains.

A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to pathwise-control the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp estimate on the stability of Markovian regime-switching processes under the perturbation of transition rate matrices is provided. Our approach is based on elaborate constructions of switching processes in the spirit of Skorokhod’s representation theorem varying according to the problem being dealt with. In particular, this method can cope with switching processes in an infinite state space and not necessarily of birth–death type. As an application, some known results on the ergodicity and stability of state-dependent regime-switching processes can be improved.

We consider an SIR (susceptible $\to$ infective $\to$ recovered) epidemic in a closed population of size n, in which infection spreads via mixing events, comprising individuals chosen uniformly at random from the population, which occur at the points of a Poisson process. This contrasts sharply with most epidemic models, in which infection is spread purely by pairwise interaction. A sequence of epidemic processes, indexed by n, and an approximating branching process are constructed on a common probability space via embedded random walks. We show that under suitable conditions the process of infectives in the epidemic process converges almost surely to the branching process. This leads to a threshold theorem for the epidemic process, where a major outbreak is defined as one that infects at least $\log n$ individuals. We show further that there exists $\delta \gt 0$, depending on the model parameters, such that the probability that a major outbreak has size at least $\delta n$ tends to one as $n \to \infty$.

In this paper, we propose new Metropolis–Hastings and simulated annealing algorithms on a finite state space via modifying the energy landscape. The core idea of landscape modification rests on introducing a parameter c, such that the landscape is modified once the algorithm is above this threshold parameter to encourage exploration, while the original landscape is utilized when the algorithm is below the threshold for exploitation purposes. We illustrate the power and benefits of landscape modification by investigating its effect on the classical Curie–Weiss model with Glauber dynamics and external magnetic field in the subcritical regime. This leads to a landscape-modified mean-field equation, and with appropriate choice of c the free energy landscape can be transformed from a double-well into a single-well landscape, while the location of the global minimum is preserved on the modified landscape. Consequently, running algorithms on the modified landscape can improve the convergence to the ground state in the Curie–Weiss model. In the setting of simulated annealing, we demonstrate that landscape modification can yield improved or even subexponential mean tunnelling time between global minima in the low-temperature regime by appropriate choice of c, and we give a convergence guarantee using an improved logarithmic cooling schedule with reduced critical height. We also discuss connections between landscape modification and other acceleration techniques, such as Catoni’s energy transformation algorithm, preconditioning, importance sampling, and quantum annealing. The technique developed in this paper is not limited to simulated annealing, but is broadly applicable to any difference-based discrete optimization algorithm by a change of landscape.

A system of interacting multi-class finite-state jump processes is analyzed. The model under consideration consists of a block-structured network with dynamically changing multi-color nodes. The interactions are local and described through local empirical measures. Two levels of heterogeneity are considered: between and within the blocks where the nodes are labeled into two types. The central nodes are those connected only to nodes from the same block, whereas the peripheral nodes are connected to both nodes from the same block and nodes from other blocks. Limits of such systems as the number of nodes tends to infinity are investigated. In particular, under specific regularity conditions, propagation of chaos and the law of large numbers are established in a multi-population setting. Moreover, it is shown that, as the number of nodes goes to infinity, the behavior of the system can be represented by the solution of a McKean–Vlasov system. Then, we prove large deviations principles for the vectors of empirical measures and the empirical processes, which extends the classical results of Dawson and Gärtner (Stochastics 20, 1987) and Léonard (Ann. Inst. H. Poincaré Prob. Statist. 31, 1995).

We consider a class of processes describing a population consisting of k types of individuals. The process is almost surely absorbed at the origin within finite time, and we study the expected time taken for such extinction to occur. We derive simple and precise asymptotic estimates for this expected persistence time, starting either from a single individual or from a quasi-equilibrium state, in the limit as a system size parameter N tends to infinity. Our process need not be a Markov process on $ {\mathbb Z}_+^k$; we allow the possibility that individuals’ lifetimes may follow more general distributions than the exponential distribution.

Consider a two-type Moran population of size N with selection and mutation, where the selective advantage of the fit individuals is amplified at extreme environmental conditions. Assume selection and mutation are weak with respect to N, and extreme environmental conditions rarely occur. We show that, as $N\to\infty$, the type frequency process with time sped up by N converges to the solution to a Wright–Fisher-type SDE with a jump term modeling the effect of the environment. We use an extension of the ancestral selection graph (ASG) to describe the genealogical picture of the model. Next, we show that the type frequency process and the line-counting process of a pruned version of the ASG satisfy a moment duality. This relation yields a characterization of the asymptotic type distribution. We characterize the ancestral type distribution using an alternative pruning of the ASG. Most of our results are stated in annealed and quenched form.

For a quadratic Markov branching process (QMBP), we show that the decay parameter is equal to the first eigenvalue of a Sturm–Liouville operator associated with the partial differential equation that the generating function of the transition probability satisfies. The proof is based on the spectral properties of the Sturm–Liouville operator. Both the upper and lower bounds of the decay parameter are given explicitly by means of a version of Hardy’s inequality. Two examples are provided to illustrate our results. The important quantity, the Hardy index, which is closely linked to the decay parameter of the QMBP, is deeply investigated and estimated.

Let f be the density function associated to a matrix-exponential distribution of parameters $(\boldsymbol{\alpha}, T,\boldsymbol{{s}})$ . By exponentially tilting f, we find a probabilistic interpretation which generalizes the one associated to phase-type distributions. More specifically, we show that for any sufficiently large $\lambda\ge 0$ , the function $x\mapsto \left(\int_0^\infty e^{-\lambda s}f(s)\textrm{d} s\right)^{-1}e^{-\lambda x}f(x)$ can be described in terms of a finite-state Markov jump process whose generator is tied to T. Finally, we show how to revert the exponential tilting in order to assign a probabilistic interpretation to f itself.

We present a study of the joint distribution of both the state of a level-dependent quasi-birth–death (QBD) process and its associated running maximum level, at a fixed time t: more specifically, we derive expressions for the Laplace transforms of transition functions that contain this information, and the expressions we derive contain familiar constructs from the classical theory of QBD processes. Indeed, one important takeaway from our results is that the distribution of the running maximum level of a level-dependent QBD process can be studied using results that are highly analogous to the more well-established theory of level-dependent QBD processes that focuses primarily on the joint distribution of the level and phase. We also explain how our methods naturally extend to the study of level-dependent Markov processes of M/G/1 type, if we instead keep track of the running minimum level instead of the running maximum level.

This paper presents a new matrix-infinite-product-form (MIP-form) solution for the stationary distribution in upper block-Hessenberg Markov chains (UBH-MCs). The existing MIP-form solution (Masuyama, Queueing Systems 92, 2019, pp. 173–200) requires a certain parameter set that satisfies both a Foster–Lyapunov drift condition and a convergence condition. In contrast, the new MIP-form solution requires no such parameter sets and no other conditions. The new MIP-form solution also has ‘quasi-algorithmic constructibility’, which is a newly introduced feature of being constructed by iterating infinitely many times a recursive procedure of finite complexity per iteration. This feature is not found in the other existing solutions for the stationary distribution in general UBH-MCs.

We consider the spectral analysis of several examples of bilateral birth–death processes and compute explicitly the spectral matrix and the corresponding orthogonal polynomials. We also use the spectral representation to study some probabilistic properties of the processes, such as recurrence, the invariant distribution (if it exists), and the probability current.

We study a general class of interacting particle systems called kinetically constrained models (KCM) in two dimensions tightly linked to the monotone cellular automata called bootstrap percolation. There are three classes of such models, the most studied being the critical one. In a recent series of works by Martinelli, Morris, Toninelli and the authors, it was shown that the KCM counterparts of critical bootstrap percolation models with the same properties split into two classes with different behaviour. Together with the companion paper by the first author, our work determines the logarithm of the infection time up to a constant factor for all critical KCM, which were previously known only up to logarithmic corrections. This improves all previous results except for the Duarte-KCM, for which we give a new proof of the best result known. We establish that on this level of precision critical KCM have to be classified into seven categories instead of the two in bootstrap percolation. In the present work, we establish lower bounds for critical KCM in a unified way, also recovering the universality result of Toninelli and the authors and the Duarte model result of Martinelli, Toninelli and the second author.

Latouche and Nguyen (2015b) constructed a sequence of stochastic fluid processes and showed that it converges weakly to a Markov-modulated Brownian motion (MMBM). Here, we construct a different sequence of stochastic fluid processes and show that it converges strongly to an MMBM. To the best of our knowledge, this is the first result on strong convergence to a Markov-modulated Brownian motion. Besides implying weak convergence, such a strong approximation constitutes a powerful tool for developing deep results for sophisticated models. Additionally, we prove that the rate of this almost sure convergence is $o(n^{-1/2} \log n)$ . When reduced to the special case of standard Brownian motion, our convergence rate is an improvement over that obtained by a different approximation in Gorostiza and Griego (1980), which is $o(n^{-1/2}(\log n)^{5/2})$ .

This paper addresses the task of modeling severity losses using segmentation when the data distribution does not fall into the usual regression frameworks. This situation is not uncommon in lines of business such as third-party liability insurance, where heavy-tails and multimodality often hamper a direct statistical analysis. We propose to use regression models based on phase-type distributions, regressing on their underlying inhomogeneous Markov intensity and using an extension of the expectation–maximization algorithm. These models are interpretable and tractable in terms of multistate processes and generalize the proportional hazards specification when the dimension of the state space is larger than 1. We show that the combination of matrix parameters, inhomogeneity transforms, and covariate information provides flexible regression models that effectively capture the entire distribution of loss severities.

We revisit the forward algorithm, developed by Irle, to characterize both the value function and the stopping set for a large class of optimal stopping problems on continuous-time Markov chains. Our objective is to renew interest in this constructive method by showing its usefulness in solving some constrained optimal stopping problems that have emerged recently.

We consider a class of phase-type distributions (PH-distributions), to be called the MMPP class of PH-distributions, and find bounds of their mean and squared coefficient of variation (SCV). As an application, we have shown that the SCV of the event-stationary inter-event time for Markov modulated Poisson processes (MMPPs) is greater than or equal to unity, which answers an open problem for MMPPs. The results are useful for selecting proper PH-distributions and counting processes in stochastic modeling.