Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-10T07:06:41.213Z Has data issue: false hasContentIssue false
  • English
  • Français

Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes

Part of: Markov processes

Published online by Cambridge University Press:  01 July 2016

Sean P. Meyn  and
R. L. Tweedie
Sean P. Meyn*
Affiliation:
University of Illinois
R. L. Tweedie*
Affiliation:
Colorado State University
*
∗ Postal address: University of Illinois, Coordinated Science Laboratory, 1101 W. Springfield Ave., Urbana, IL 61801, USA.
∗∗ Postal address: Department of Statistics, Colorado State University, Fort Collins, CO 80523, USA.
Get access Rights & Permissions [Opens in a new window]

Abstract

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator.

Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula.

We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case.

We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

Keywords

FOSTER'S CRITERIONIRREDUCIBLE MARKOV PROCESSESSTOCHASTIC LYAPUNOVFUNCTIONSERGODICITYEXPONENTIAL ERGODICITYRECURRENCESTORAGE MODELSRISK MODELSQUEUESHYPOELLIPTIC DIFFUSION

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Type
Research Article
Information
Advances in Applied Probability , Volume 25 , Issue 3 , September 1993 , pp. 518 - 548
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This work was commenced at Bond University, and developed there, at the Australian National University, the University of Illinois, and Colorado State University.

Work supported in part by NSF initiation grant #ECS 8910088.

References

[1] Azéma, J. Kaplan, Duflo and Revuz, D. (1967) Measure invariante sur les classes récurrentes des processus de Markov. Z. Wahrscheinlichkeitsth. 8, 157–181.CrossRefGoogle Scholar
[2] Brockwell, P. J. (1977) Stationary distribution for dams with additive input and content-dependent release rate. Adv. Appl. Prob. 9, 645–663.CrossRefGoogle Scholar
[3] Brockwell, P. J. Resnick, S. J. and Tweedie, R. L. (1982) Storage processes with general release rule and additive inputs. Adv. App. Prob. 14, 392–433.Google Scholar
[4] Browne, S. and Sigman, K. (1992) Work-modulated queues with applications to storage processes. J. Appl. Prob. 29, 699–712.CrossRefGoogle Scholar
[5] Chen, M.-F. (1986) Jump Processes and Particle Systems. Beijing Normal University Press, Beijing, China.Google Scholar
[6] Chen, M.-F. (1991) On three classical problems for Markov chains with continuous time parameter. J. Appl. Prob. 28, 305–320.Google Scholar
[7] ÇInlar, E. and Pinsky, M. (1972) On dams with additive inputs and general release rule. J. Appl. Prob. 9, 472–492.Google Scholar
[8] Davis, M. H. A. (1984) Piecewise-deterministic Markov processes: A general class of nondiffusion stochastic models. J. R. Statist. Soc. B. 46, 353–388.Google Scholar
[9] Davis, M. H. A. (1993) Markov Models and Optimization. Chapman and Hall, London.Google Scholar
[10] Doob, J. L. (1953) Stochastic Processes. Wiley, New York.Google Scholar
[11] Foguel, S. R. (1969 ) Positive operators on C(X). Proc. Amer. Math. Soc. 22, 295–297.Google Scholar
[12] Getoor, R. K. (1979) Transience and recurrence of Markov processes. In Séminaire de Probabilités XVI, pp. 397–409, ed. Azéma, J. and Yor, M. Springer-Verlag, Berlin.Google Scholar
[13] Harrison, J. M. and Resnick, S. I. (1976) The stationary distribution and first exit probabilities of a storage process with general release rule. Math. Oper. Res. 1, 347–358.Google Scholar
[14] Harrison, J. M. and Resnick, S. I. (1978) The recurrence classification of risk and storage processes. Math. Operat. Res. 3, 57–66.Google Scholar
[15] Hordijk, A. and Spieksma, F. M. (1992) On ergodicity and recurrence properties of a Markov chain with an application. Adv. Appl. Prob. 24, 343–376.Google Scholar
[16] Kalashnikov, V. V. (1975) Certain properties of piecewise-linear Markov processes. Theory Prob. Appl. XX, 560–571.Google Scholar
[17] Khas'Minskii, R. Z. (1980) Stochastic Stability of Differential Equations, Volume 7 of Mechanics–Analysis. Sijthoff & Noordhoff, Rockville, Md.Google Scholar
[18] Kliemann, W. (1987) Recurrence and invariant measures for degenerate diffusions. Ann. Prob. 15, 690–707.CrossRefGoogle Scholar
[19] Kushner, H. J. (1967) Stochastic Stability and ControlVolume 33 of Mathematics in Science and Engineering. Academic Press, New York.Google Scholar
[20] Lamperti, J. (1960) Criteria for the recurrence or transience of stochastic processes I. J. Math. Anal. Appl. 1, 314–330.CrossRefGoogle Scholar
[21] Meyn, S. P. and Down, D. (1993) Stability of generalized Jackson networks. Ann. Appl. Prob. CrossRefGoogle Scholar
[22] Meyn, S. P. and Tweedie, R. L. (1992a) Generalized Resolvents and Harris Recurrence of Markov Processes. Contemporary Mathematics. American Mathematical Society, Providence, RI.Google Scholar
[23] Meyn, S. P. and Tweedie, R. L. (1993a) Markov Chains and Stochastic Stability. Control and Communication in Engineering. Springer-Verlag, Berlin.CrossRefGoogle Scholar
[24] Meyn, S. P. and Tweedie, R. L. (1992b) Stability of Markovian processes I: discrete time chains. Adv. Appl. Prob. 24, 542–574.Google Scholar
[25] Meyn, S. P. and Tweedie, R. L. (1993b) Stability of Markovian processes II: Continuous time processes and sampled chains. Adv. Appl. Prob. 25, 487–517.Google Scholar
[26] Nummelin, E. (1984) General Irreducible Markov Chains and Non-negative Operators. Cambridge University Press.Google Scholar
[27] Nummelin, E. and Tuominen, P. (1982) Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory. Stoch. Proc. Appl. 12, 187–202.Google Scholar
[28] Sharpe, M. (1988) General Theory of Markov Processes. Academic Press, New York.Google Scholar
[29] Spieksma, F. M. (1991) Geometrically Ergodic Markov Chains and the Optimal Control of Queues. PhD Thesis, University of Leiden.Google Scholar
[30] Stettner, L. (1986) On the existence and uniqueness of invariant measure for continuous time Markov processes. Technical Report LCDS #86-18, Brown University, Providence, RI, April.Google Scholar
[31] Tuominen, P. and Tweedie, R. L. (1979a). Exponential decay and ergodicity of general Markov processes and their discrete skeletons. Adv. Appl. Prob. 11, 784–803.CrossRefGoogle Scholar
[32] Tuominen, P. and Tweedie, R. L. (1979b) Exponential ergodicity in Markovian queueing and dam models. J. Appl. Prob. 16, 867–880.CrossRefGoogle Scholar
[33] Tweedie, R. L. (1975a) Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space. Stoch. Proc. Appl. 3, 385–403.CrossRefGoogle Scholar
[34] Tweedie, R. L. (1975b) Sufficient conditions for regularity, recurrence and ergodicity of Markov processes. Math. Proc. Camb. Phil. Soc. 78, 125–136.CrossRefGoogle Scholar
[35] Tweedie, R. L. (1976) Criteria for classifying general Markov chains. Adv. Appl. Prob. 8, 737–771.CrossRefGoogle Scholar
[36] Tweedie, R. L. (1981) Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes. J. Appl. Prob. 18, 122–130.Google Scholar
[37] Tweedie, R. L. (1988) Invariant measures for Markov chains with no irreducibility assumptions. J. Appl. Prob. 25A, 275–285.CrossRefGoogle Scholar
[38] Wonham, W. M. (1966) A Liapunov criteria for weak stochastic stability. J. Differential Eqns 2, 195–207.CrossRefGoogle Scholar