A computable necessary and sufficient condition of separation cutoff is obtained for a sequence of continuous-time upward skip-free chains with the stochastically monotone time-reversals.

We introduce a notion of kth order stochastic monotonicity and duality that allows us to unify the notion used in insurance mathematics (sometimes refereed to as Siegmund's duality) for the study of ruin probability and the duality responsible for the so-called put-call symmetries in option pricing. Our general kth order duality can be interpreted financially as put-call symmetry for powered options. The main objective of this paper is to develop an effective analytic approach to the analysis of duality that will lead to the full characterization of kth order duality of Markov processes in terms of their generators, which is new even for the well-studied case of put-call symmetries.

For exponential open and closed queueing networks, we investigate the internal dependence structure, compare the internal dependence for different networks, and discuss the relation of correlation formulae to the existence of spectral gaps and comparison of asymptotic variances. A central prerequisite for the derived theorems is stochastic monotonicity of the networks. The dependence structure of network processes is described by concordance order with respect to various classes of functions. Different networks with the same first-order characteristics are compared with respect to their second-order properties. If a network is perturbed by changing the routeing in a way which holds the routeing equilibrium fixed, the resulting perturbations of the network processes are evaluated.

A comparison theorem is stated for Markov processes in Polish state spaces. We consider a general class of stochastic orderings induced by a cone of real functions. The main result states that stochastic monotonicity of one process and comparability of the infinitesimal generators imply ordering of the processes. Several applications to convex type and to dependence orderings are given. In particular, Liggett's theorem on the association of Markov processes is a consequence of this comparison result.

We compare dependence in stochastically monotone Markov processes with partially ordered Polish state spaces using the concordance and supermodular orders. We show necessary and sufficient conditions for the concordance order to hold both in terms of the one-step transition probabilities for discrete-time processes and in terms of the corresponding infinitesimal generators for continuous-time processes. We give examples showing that a stochastic monotonicity assumption is not necessary for such orderings. We indicate relations between dependence orderings and, variously, the asymptotic variance-reduction effect in Monte Carlo Markov chains, Cheeger constants, and positive dependence for Markov processes.

We model the evolution of the credit migration of a corporate bond as an inhomogeneous semi-Markov chain. The valuation of a defaultable bond is done with the use of the forward probability of no default up to maturity time. It is proved that, under the forward probability measure, the semi-Markov property is maintained. We find the functional relationships between the forward transition probability sequences and the real-world probability sequences. The stochastic monotonicity properties of the inhomogeneous semi-Markov model, which play a prominent role in these issues, are studied in detail. Finally, we study the term structure of credit spread, provide an algorithm for the estimation of the forward probabilities of transitions under the risk premium assumptions, and present an estimation method for the real-world probability sequences.

We consider Markov chains in the context of iterated random functions and show the existence and uniqueness of an invariant distribution under a local contraction condition combined with a drift condition, extending results of Diaconis and Freedman. From these we deduce various other topological stability properties of the chains. Our conditions are typically satisfied by, for example, queueing and storage models where the global Lipschitz condition used by Diaconis and Freedman normally fails.

In this paper we investigate the application of perfect simulation, in particular Coupling from the Past (CFTP), to the simulation of random point processes. We give a general formulation of the method of dominated CFTP and apply it to the problem of perfect simulation of general locally stable point processes as equilibrium distributions of spatial birth-and-death processes. We then investigate discrete-time Metropolis-Hastings samplers for point processes, and show how a variant which samples systematically from cells can be converted into a perfect version. An application is given to the Strauss point process.

In this paper we give bounds on the total variation distance from convergence of a continuous time positive recurrent Markov process on an arbitrary state space, based on Foster-Lyapunov drift and minorisation conditions. Considerably improved bounds are given in the stochastically monotone case, for both discrete and continuous time models, even in the absence of a reachable minimal element. These results are applied to storage models and to diffusion processes.

In this paper we consider an irreducible random walk in Z+ defined by X(m+1) = max(0, X(m) + A(m+1)) with E{A} < 0 and for an s ≥ 0 where a+ = max(0,a). Let π be the stationary distribution of X. We show that one can find probability distributions πn supported by {0,n} such that ||πn - π||1 ≤ Cn-s, where the constant C is computable in terms of the moments of A, and also that ||πn - π||1 = o(n-s). Moreover, this upper bound reveals exact for s ≥ 1, in the sense that, for any positive ε, we can find a random walk fulfilling the above assumptions and for which the relation ||πn - π||1 = o(n-s-ε) does not hold. This result is used to derive the exact convergence rate of the time stationary distribution of an M/GI/1/n queueing system to the time stationary distribution of the corresponding M/GI/1 queueing system when n tends to infinity.

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n)P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n)P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

Using a matrix approach we discuss the first-passage time of a Markov process to exceed a given threshold or for the maximal increment of this process to pass a certain critical value. Conditions under which this first-passage time possesses various ageing properties are studied. Some results previously obtained by Li and Shaked (1995) are extended.

Let P be an infinite irreducible stochastic matrix, recurrent positive and stochastically monotone and Pn be any n × n stochastic matrix with Pn ≧ Tn, where Tn denotes the n × n northwest corner truncation of P. These assumptions imply the existence of limit distributions π and πn for P and Pn respectively. We show that if the Markov chain with transition probability matrix P meets the further condition of geometric recurrence then the exact convergence rate of πn to π can be expressed in terms of the radius of convergence of the generating function of π. As an application of the preceding result, we deal with the random walk on a half line and prove that the assumption of geometric recurrence can be relaxed. We also show that if the i.i.d. input sequence (A(m)) is such that we can find a real number r0 > 1 with , then the exact convergence rate of πn to π is characterized by r0. Moreover, when the generating function of A is not defined for |z| > 1, we derive an upper bound for the distance between πn and π based on the moments of A.

We examine the existence of limiting behavior, or stability, for storage models with shot noise input and general release rules. The shot noise feature of the input process allows the individual inputs to gradually enter the store.

We first show that a store under the unit release rule is stable if and only if the traffic intensity is less than one; this extends the classic result of Prabhu (1980) to the case of shot noise input. The stability of the unit release rule store and various stochastic orderings are then used to derive a sufficient condition for a store with a general release rule to be stable. Finally, we show that when restricted to a compact state space, our storage model is always stable.

An important component of the paper is the methodology employed: coupling and stochastic monotonicity play a key role in analyzing the non-Markov processes encountered.

In this paper, we study a repair replacement model for a stochastically deteriorating system. For the expected discounted reward case, we show that the optimal replacement policy is of the form ‘replace at the time of the Nth failure'.

In the multi-armed bandit problem, the decision-maker must choose each period a single project to work on. From the chosen project she receives an immediate reward that depends on the current state of the project. Next period the chosen project makes a stochastic transition to a new state, but projects that are not chosen remain in the same state. What happens in a two-armed bandit context if projects not chosen do not remain in the same state? We derive two sufficient conditions for the optimal policy to be myopic: either the transition function for chosen projects has in a certain sense uniformly stronger stochastic dominance than the transition function for unchosen projects, or both transition processes are normal martingales, the variance of which is independent of the history of process choices.

Let {N(t)} be a renewal process having the associated age process {X(t)}. Of interest is the counting process {M(t)} characterized by a non-homogeneous Poisson process with age-dependent intensity function λ (X(t)). The trivariate process {Y(t) = [M(t), N(t), X(t)]} is analyzed obtaining its Laplace transform generating function explicitly. Based on this result, asymptotic behavior of {S(t) = cM(t) + dN(t)} as t → ∞ is discussed. Furthermore, a sufficient condition is given under which {M(t), –N(t), X(t)} is stochastically monotone and associated. This condition also assures increasing stochastic convexity of {M(t)}. The usefulness of these results is demonstrated through an application to the age-dependent minimal repair problem.

In this note, we study a new repair replacement model for a deteriorating system, in which the successive survival times of the system form a geometric process and are stochastically non-increasing, whereas the consecutive repair times after failure also constitute a geometric process but are stochastically non-decreasing. Two kinds of replacement policy are considered, one based on the working age of the system and the other one determined by the number of failures. The explicit expressions of the long-run average costs per unit time under these two kinds of policy are calculated.

Several notions of stochastic convexity and concavity and their properties are studied in this paper. Efficient sample path approaches are developed in order to verify the occurrence of these notions in various applications. Numerous examples are given. The use of these notions in several areas of probability and statistics is demonstrated. In queueing theory, the convexity (as a function of c) of the steady-state waiting time in a GI/D/c queue, and (as a function of the arrival or service rates) in a GI/G/1 queue, is established. Also the convexity of the queue length in the M/M/c case (as a function of the arrival rate) is shown, thus strengthening previous results while simplifying their derivation. In reliability theory, the convexity of the payoff in the success rate of an imperfect repair is obtained and used to find an optimal repair probability. Also the convexity of the damage as a function of time in a cumulative damage shock model is shown. In branching processes, the convexity of the population size as a function of a parameter of the offspring distribution is proved. In non-parametric statistics, the stochastic concavity (or convexity) of the empirical distribution function is established. And, for applications in the theory of probability inequalities, we identify several families of distributions which are convexly parametrized.

A general method is developed to compute easy bounds of the weighted stationary probabilities for networks of queues which do not satisfy the standard product form. The bounds are obtained by constructing approximating reversible Markov chains. Thus, the bounds are insensitive with respect to service-time distributions. A special representation, called the tree-form solution, of the stationary distribution is used to derive the bounds. The results are applied to an overflow model.