In this work, we study discrete-time Markov decision processes (MDPs) with constraints when all the objectives have the same form of expected total cost over the infinite time horizon. Our objective is to analyze this problem by using the linear programming approach. Under some technical hypotheses, it is shown that if there exists an optimal solution for the associated linear program then there exists a randomized stationary policy which is optimal for the MDP, and that the optimal value of the linear program coincides with the optimal value of the constrained control problem. A second important result states that the set of randomized stationary policies provides a sufficient set for solving this MDP. It is important to note that, in contrast with the classical results of the literature, we do not assume the MDP to be transient or absorbing. More importantly, we do not impose the cost functions to be nonnegative or to be bounded below. Several examples are presented to illustrate our results.

In this paper we investigate the asymptotic behaviors of the finite- and infinite-time ruin probabilities for a Poisson risk model with stochastic investment returns which constitute a general adapted càdlàg process and heavy-tailed claim sizes which are bivariate upper tail independent. The results of this paper show that the asymptotic ruin probabilities are dominated by the extreme of insurance risk but not by that of investment risk. As applications of the results, we discuss four special cases when the investment returns are determined by a fractional Brownian motion, an integrated Vasicek model, an integrated Cox–Ingersoll–Ross model, and the Heston model.

Spot prices in energy markets exhibit special features, such as price spikes, mean reversion, stochastic volatility, inverse leverage effect, and dependencies between the commodities. In this paper a multivariate stochastic volatility model is introduced which captures these features. The second-order structure and stationarity of the model are analyzed in detail. A simulation method for Monte Carlo generation of price paths is introduced and a numerical example is presented.

Many regenerative arguments in stochastic processes use random times which are akin to stopping times, but which are determined by the future as well as the past behaviour of the process of interest. Such arguments based on ‘conditioning on the future’ are usually developed in an ad-hoc way in the context of the application under consideration, thereby obscuring the underlying structure. In this paper we give a simple, unified, and more general treatment of such conditioning theory. We further give a number of novel applications to various particle system models, in particular to various flavours of contact processes and to infinite-bin models. We give a number of new results for existing and new models. We further make connections with the theory of Harris ergodicity.

In Benth and Vos (2013) we introduced a multivariate spot price model with stochastic volatility for energy markets which captures characteristic features, such as price spikes, mean reversion, stochastic volatility, and inverse leverage effect as well as dependencies between commodities. In this paper we derive the forward price dynamics based on our multivariate spot price model, providing a very flexible structure for the forward curves, including contango, backwardation, and hump shape. Moreover, a Fourier transform-based method to price options on the forward is described.

The Halfin–Whitt regime, or the quality-and-efficiency-driven (QED) regime, for multiserver systems refers to a situation with many servers, a critical load, and yet favorable system performance. We apply this regime to the classical multiserver loss system with slow retrials. We derive nondegenerate limiting expressions for the main steady-state performance measures, including the retrial rate and the blocking probability. It is shown that the economies of scale associated with the QED regime persist for systems with retrials, although in situations when the load becomes extremely critical the retrials cause deteriorated performance. Most of our results are obtained by a detailed analysis of Cohen's equation that defines the retrial rate in an implicit way. The limiting expressions are established by studying prelimit behavior and exploiting the connection between Cohen's equation and Mills' ratio for the Gaussian and Poisson distributions.

We extend many of the classical results for standard one-dimensional diffusions to a diffusion process with memory of the form d Xt=σ(Xt,Xt)dWt, where Xt= m ∧ inf0 ≤s≤tXs. In particular, we compute the expected time for X to leave an interval, classify the boundary behavior at 0, and derive a new occupation time formula for X. We also show that (Xt,Xt) admits a joint density, which can be characterized in terms of two independent tied-down Brownian meanders (or, equivalently, two independent Bessel-3 bridges). Finally, we show that the joint density satisfies a generalized forward Kolmogorov equation in a weak sense, and we derive a new forward equation for down-and-out call options.

We consider a random trial-based telegraph process, which describes a motion on the real line with two constant velocities along opposite directions. At each epoch of the underlying counting process the new velocity is determined by the outcome of a random trial. Two schemes are taken into account: Bernoulli trials and classical Pólya urn trials. We investigate the probability law of the process and the mean of the velocity of the moving particle. We finally discuss two cases of interest: (i) the case of Bernoulli trials and intertimes having exponential distributions with linear rates (in which, interestingly, the process exhibits a logistic stationary density with nonzero mean), and (ii) the case of Pólya trials and intertimes having first gamma and then exponential distributions with constant rates.

For a family of linear preferential attachment graphs, we provide rates of convergence for the total variation distance between the degree of a randomly chosen vertex and an appropriate power law distribution as the number of vertices tends to ∞. Our proof uses a new formulation of Stein's method for the negative binomial distribution, which stems from a distributional transformation that has the negative binomial distributions as the only fixed points.

We consider the problem of reducing the response time of fork-join systems by maintaining the workload balanced among the processing stations. The general problem of modeling and finding an optimal policy that reduces imbalance is quite difficult. In order to circumvent this difficulty, the heavy traffic approach is taken, and the system dynamics are approximated by a reflected diffusion process. This way, the problem of finding an optimal balancing policy that reduces workload imbalance is set as a stochastic optimal control problem, for which numerical methods are available. Some numerical experiments are presented, where the control problem is solved numerically and applied to a simulation. The results indicate that the response time of the controlled system is reduced significantly using the devised control.

The objective of this paper is to give a rigorous analysis of a stochastic spatial model of producer-consumer systems that has been recently introduced by Kang and the author to understand the role of space in ecological communities in which individuals compete for resources. Each point of the square lattice is occupied by an individual which is characterized by one of two possible types, and updates its type in continuous time at rate 1. Each individual being thought of as a producer and consumer of resources, the new type at each update is chosen at random from a certain interaction neighborhood according to probabilities proportional to the ability of the neighbors to consume the resource produced by the individual to be updated. In addition to giving a complete qualitative picture of the phase diagram of the spatial model, our results indicate that the nonspatial deterministic mean-field approximation of the stochastic process fails to describe the behavior of the system in the presence of local interactions. In particular, we prove that, in the parameter region where the nonspatial model displays bistability, there is a dominant type that wins regardless of its initial density in the spatial model, and that the inclusion of space also translates into a significant reduction of the parameter region where both types coexist.

This article is concerned with a stochastic multipatch model in which each local population is subject to a strong Allee effect. The model is obtained by using the framework of interacting particle systems to extend a stochastic two-patch model that was recently introduced by Kang and the author. The main objective is to understand the effect of the geometry of the network of interactions, which represents potential migrations between patches, on the long-term behavior of the metapopulation. In the limit as the number of patches tends to ∞, there is a critical value for the Allee threshold below which the metapopulation expands and above which the metapopulation goes extinct. Spatial simulations on large regular graphs suggest that this critical value strongly depends on the initial distribution when the degree of the network is large, whereas the critical value does not depend on the initial distribution when the degree is small. Looking at the system starting with a single occupied patch on the complete graph and on the ring, we prove analytical results that support this conjecture. From an ecological perspective, these results indicate that, upon arrival of an alien species subject to a strong Allee effect to a new area, though dispersal is necessary for its expansion, fast long-range dispersal drives the population toward extinction.