Hostname: page-component-78c5997874-v9fdk Total loading time: 0 Render date: 2024-11-10T15:32:31.241Z Has data issue: false hasContentIssue false
  • English
  • Français

Exponential ergodicity and steady-state approximations for a class of markov processes under fast regime switching

Part of: Hamilton-Jacobi theories, including dynamic programming Special processes Limit theorems Operations research and management science

Published online by Cambridge University Press:  17 March 2021

Ari Arapostathis ,
Guodong Pang  and
Yi Zheng
Ari Arapostathis*
Affiliation:
The University of Texas at Austin
Guodong Pang*
Affiliation:
Pennsylvania State University
Yi Zheng*
Affiliation:
Pennsylvania State University
*
*Postal address: Department of Electrical and Computer Engineering, The University of Texas at Austin, 2501 Speedway, EERC 7.824, Austin, TX78712. Email address: ari@utexas.edu
**Postal address: The Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, College of Engineering, Pennsylvania State University, University Park, PA16802.
**Postal address: The Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, College of Engineering, Pennsylvania State University, University Park, PA16802.
Get access Rights & Permissions [Opens in a new window]

Abstract

We study ergodic properties of a class of Markov-modulated general birth–death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that is taken to be large. Under very weak hypotheses, we show that if the averaged process is exponentially ergodic for large values of the parameter, then the same applies to the original joint Markov process. The second set of results concerns steady-state diffusion approximations, under the assumption that the ‘averaged’ fluid limit exists. Here, we establish convergence rates for the moments of the approximating diffusion process to those of the Markov-modulated birth–death process. This is accomplished by comparing the generator of the approximating diffusion and that of the joint Markov process. We also provide several examples which demonstrate how the theory can be applied.

Keywords

Markov-modulated processbirth–death processexponential ergodicitysteady-state approximationsfast regime switching

MSC classification

Primary: 60K25: Queueing theory 90B20: Traffic problems
Secondary: 90B36: Scheduling theory, stochastic 49L20: Dynamic programming method 60F17: Functional limit theorems; invariance principles
Type
Original Article
Information
Advances in Applied Probability , Volume 53 , Issue 1 , March 2021 , pp. 1 - 29
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, D., Blom, J., Mandjes, M., Thorsdottir, H. and De Turck, K. (2016). A functional central limit theorem for a Markov-modulated infinite-server queue. Methodology Comput. Appl. Prob. 18, 153–168.CrossRefGoogle Scholar
Arapostathis, A., Biswas, A. and Pang, G. (2015). Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime. Ann. Appl. Prob. 25, 3511–3570.CrossRefGoogle Scholar
Arapostathis, A., Das, A., Pang, G. and Zheng, Y. (2019). Optimal control of Markov-modulated multiclass many-server queues. Stoch. Systems 9, 155–181.CrossRefGoogle Scholar
Arapostathis, A., Hmedi, H. and Pang, G. (2020). On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin–Whitt regime. To appear in Math. Operat. Res. Google Scholar
Arapostathis, A., Pang, G. and Sandrić, N. (2019). Ergodicity of a Lévy-driven SDE arising from multiclass many-server queues. Ann. Appl. Prob. 29, 1070–1126.CrossRefGoogle Scholar
Billingsley, P. (1999). Convergence of Probability Measures, 2nd edn. John Wiley, New York.CrossRefGoogle Scholar
Coolen-Schrijner, P. and van Doorn, E. A. (2002). The deviation matrix of a continuous-time Markov chain. Prob. Eng. Inf. Sci. 16, 351–366.10.1017/S0269964802163066CrossRefGoogle Scholar
Dai, J. G., He, S. and Tezcan, T. (2010). Many-server diffusion limits for $G/PH/n+GI$ queues. Ann. Appl. Prob. 20, 1854–1890.CrossRefGoogle Scholar
Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Prob. 23, 1291–1317.10.1214/12-AAP870CrossRefGoogle Scholar
Fralix, B. H. and Adan, I. J. B. F. (2009). An infinite-server queue influenced by a semi-Markovian environment. Queueing Systems 61, 65–84.CrossRefGoogle Scholar
Gurvich, I. (2014). Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Prob. 24, 2527–2559.CrossRefGoogle Scholar
Hmedi, H., Arapostathis, A. and Pang, G. (2019). Uniform stability of a class of large-scale parallel server networks. Preprint. Available at https://arxiv.org/abs/1907.04793.Google Scholar
Hunter, J. J. (1982). Generalized inverses and their application to applied probability problems. Linear Algebra Appl. 45, 157–198.CrossRefGoogle Scholar
Jansen, H. M., Mandjes, M., De Turck, K. and Wittevrongel, S. (2019). Diffusion limits for networks of Markov-modulated infinite-server queues. Performance Evaluation 135.CrossRefGoogle Scholar
Jonckheere, M. and Shneer, S. (2014). Stability of multi-dimensional birth-and-death processes with state-dependent 0-homogeneous jumps. Adv. Appl. Prob. 46, 59–75.CrossRefGoogle Scholar
Khasminskii, R. Z. (2012). Stability of regime-switching stochastic differential equations. Probl. Inf. Transm. 48, 259–270.CrossRefGoogle Scholar
Khasminskii, R. Z., Zhu, C. and Yin, G. (2007). Stability of regime-switching diffusions. Stoch. Process. Appl. 117, 1037–1051.CrossRefGoogle Scholar
Kumar, R., Lewis, M. E. and Topaloglu, H. (2013). Dynamic service rate control for a single-server queue with Markov-modulated arrivals. Naval Res. Logistics 60, 661–677.10.1002/nav.21560CrossRefGoogle Scholar
Mao, X. (1999). Stability of stochastic differential equations with Markovian switching. Stoch. Process. Appl. 79, 45–67.CrossRefGoogle Scholar
Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518–548.CrossRefGoogle Scholar
Pang, G., Talreja, R. and Whitt, W. (2007). Martingale proofs of many-server heavy-traffic limits for Markovian queues. Prob. Surveys 4, 193–267.CrossRefGoogle Scholar
Pang, G. and Zheng, Y. (2017). On the functional and local limit theorems for Markov modulated compound Poisson processes. Statist. Prob. Lett. 129, 131–140.CrossRefGoogle Scholar
Shao, J. and Xi, F. (2014). Stability and recurrence of regime-switching diffusion processes. SIAM J. Control Optimization 52, 3496–3516.CrossRefGoogle Scholar
Xia, L., He, Q. and Alfa, A. S. (2017). Optimal control of state-dependent service rates in a MAP/M/1 queue. IEEE Trans. Automatic Control 62, 4965–4979.CrossRefGoogle Scholar
Zeifman, A. I. (1998). Stability of birth-and-death processes. J. Math. Sci. 91, 3023–3031.CrossRefGoogle Scholar
Zhu, Y. and Prabhu, N. U. (1991). Markov-modulated $PH/G/1$ queueing systems. Queueing Systems 9, 313–322.CrossRefGoogle Scholar