Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-28T01:59:59.411Z Has data issue: false hasContentIssue false
  • English
  • Français

Analysis of non-reversible Markov chains via similarity orbits

Part of: Markov processes Ergodic theory

Published online by Cambridge University Press:  18 February 2020

Michael C. H. Choi  and
Pierre Patie
Michael C. H. Choi*
Affiliation:
Institute for Data and Decision Analytics, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, PR China and Shenzhen Institute of Artificial Intelligence and Robotics for Society
Pierre Patie
Affiliation:
School of Operations Research and Information Engineering, Cornell University, Ithaca, NY 14853, USA and Laboratoire de Mathématiques et leurs Applications, UMR CNRS 5142, Université de Pau et des Pays de l’Adour, Pau, France
*
*Corresponding author. Email: michaelchoi@cuhk.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity orbit of a normal transition kernel, such as that of birth–death chains or reversible Markov chains. We start by identifying a set of sufficient conditions for a Markov chain to belong to the similarity orbit of a birth–death chain. As by-products, we obtain a spectral representation in terms of non-self-adjoint resolutions of identity in the sense of Dunford [21] and offer a detailed analysis on the convergence rate, separation cutoff and L2-cutoff of this class of non-reversible Markov chains. We also look into the problem of estimating the integral functionals from discrete observations for this class. In the last part of this paper we investigate a particular similarity orbit of reversible Markov kernels, which we call the pure birth orbit, and analyse various possibly non-reversible variants of classical birth–death processes in this orbit.

MSC classification

Primary: 60J05: Discrete-time Markov processes on general state spaces 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) 37A25: Ergodicity, mixing, rates of mixing 37A30: Ergodic theorems, spectral theory, Markov operators
Type
Paper
Information
Combinatorics, Probability and Computing , Volume 29 , Issue 4 , July 2020 , pp. 508 - 536
Check for updates
Copyright
© Cambridge University Press 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aldous, D. and Fill, J. A. (2002) Reversible Markov chains and random walks on graphs. Unfinished monograph, recompiled 2014. http://www.stat.berkeley.edu/∼aldous/RWG/book.htmlGoogle Scholar
Altmeyer, R. and Chorowski, J. (2018) Estimation error for occupation time functionals of stationary Markov processes. Stoch. Process. Appl. 128 1830–1848.CrossRefGoogle Scholar
Antoine, J.-P. and Trapani, C. (2013) Partial inner product spaces, metric operators and generalized Hermiticity. J. Phys. A 46 025204.CrossRefGoogle Scholar
Asmussen, S. (2003) Applied Probability and Queues, second edition, Vol. 51 of Applications of Mathematics (New York): Stochastic Modelling and Applied Probability, Springer.Google Scholar
Berman, A. and Plemmons, R. J. (1974) Matrix group monotonicity. Proc. Amer. Math. Soc. 46 355–359.Google Scholar
Bierkens, J. (2016) Non-reversible Metropolis–Hastings. Statist. Comput. 26 1213–1228.CrossRefGoogle Scholar
Chafaï, D. and Joulin, A. (2013) Intertwining and commutation relations for birth–death processes. Bernoulli 19 1855–1879.CrossRefGoogle Scholar
Chen, G.-Y., Hsu, J.-M. and Sheu, Y.-C. (2017) The L2-cutoffs for reversible Markov chains. Ann. Appl. Probab. 4 2305–2341.CrossRefGoogle Scholar
Chen, G.-Y. and Saloff-Coste, L. (2008) The cutoff phenomenon for ergodic Markov processes. Electron. J. Probab. 13 26–78.CrossRefGoogle Scholar
Chen, G.-Y. and Saloff-Coste, L. (2010) The L 2-cutoff for reversible Markov processes. J. Funct. Anal. 258 2246–2315.CrossRefGoogle Scholar
Chen, G.-Y. and Saloff-Coste, L. (2015) Computing cutoff times of birth and death chains. Electron. J. Probab. 20 76.CrossRefGoogle Scholar
Chesney, M., Jeanblanc-Picqué, M. and Yor, M. (1997) Brownian excursions and Parisian barrier options. Adv. Appl. Probab. 29 165–184.CrossRefGoogle Scholar
Choi, M. and Patie, P. (2019) Skip-free Markov chains. Trans. Amer. Math. Soc. 371 7301–7342.CrossRefGoogle Scholar
Clifford, P. and Sudbury, A. (1985) A sample path proof of the duality for stochastically monotone Markov processes. Ann. Probab. 13 558–565.CrossRefGoogle Scholar
Cloez, B. and Delplancke, C. (2019) Intertwinings and Stein’s magic factors for birth-death processes. Ann. Inst. Henri Poincaré Probab. Stat. 55 341–377.CrossRefGoogle Scholar
Cui, H. and Mao, Y.-H. (2010) Eigentime identity for asymmetric finite Markov chains. Front. Math. China 5 623–634.CrossRefGoogle Scholar
Diaconis, P. and Fill, J. A. (1990) Strong stationary times via a new form of duality. Ann. Probab. 18 1483–1522.CrossRefGoogle Scholar
Diaconis, P., Khare, K. and Saloff-Coste, L. (2008) Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci. 23 151–178.CrossRefGoogle Scholar
Diaconis, P. and Miclo, L. (2015) On quantitative convergence to quasi-stationarity. Ann. Fac. Sci. Toulouse Math. (6) 24 973–1016.CrossRefGoogle Scholar
Diaconis, P. and Saloff-Coste, L. (2006) Separation cut-offs for birth and death chains. Ann. Appl. Probab. 16 2098–2122.CrossRefGoogle Scholar
Dunford, N. (1954) Spectral operators. Pacific J. Math. 4 321–354.CrossRefGoogle Scholar
Dunford, N. and Schwartz, J. T. (1971) Linear Operators, III: Spectral Operators, Interscience (Wiley).Google Scholar
Dym, H. and McKean, H. P. (1976) Gaussian Processes, Function Theory, and the Inverse Spectral Problem, Vol. 31 of Probability and Mathematical Statistics, Academic Press (Harcourt Brace Jovanovich).Google Scholar
Fill, J. A. (1991) Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process. Ann. Appl. Probab. 1 62–87.CrossRefGoogle Scholar
Fill, J. A. (2009) On hitting times and fastest strong stationary times for skip-free and more general chains. J. Theoret. Probab. 22 587–600.CrossRefGoogle Scholar
Fort, G., Meyn, S., Moulines, E. and Priouret, P. (2008) The ODE method for stability of skip-free Markov chains with applications to MCMC. Ann. Appl. Probab. 18 664–707.CrossRefGoogle Scholar
Friedland, S. and Melkman, A. A. (1979) On the eigenvalues of nonnegative Jacobi matrices. Linear Algebra Appl. 25 239–253.Google Scholar
Griffiths, R. (2016) Multivariate Krawtchouk polynomials and composition birth and death processes. Symmetry 8 33.CrossRefGoogle Scholar
Huillet, T. and Martinez, S. (2011) Duality and intertwining for discrete Markov kernels: Relations and examples. Adv. Appl. Probab. 43 437–460.CrossRefGoogle Scholar
Inoue, A. and Trapani, C. (2014) Non-self-adjoint resolutions of the identity and associated operators. Complex Anal. Oper. Theory 8 1531–1546.CrossRefGoogle Scholar
Jansen, S. and Kurt, N. (2014) On the notion(s) of duality for Markov processes. Probab. Surv. 11 59–120.CrossRefGoogle Scholar
Karlin, S. and McGregor, J. (1959) Coincidence properties of birth and death processes. Pacific J. Math. 9 1109–1140.CrossRefGoogle Scholar
Kendall, D. G. (1959) Unitary dilations of one-parameter semigroups of Markov transition operators, and the corresponding integral representations for Markov processes with a countable infinity of states. Proc. London Math. Soc. (3) 9 417–431.CrossRefGoogle Scholar
Khare, K. and Mukherjee, N. (2013) Convergence analysis of some multivariate Markov chains using stochastic monotonicity. Ann. Appl. Probab. 23 811–833.CrossRefGoogle Scholar
Khare, K. and Zhou, H. (2009) Rates of convergence of some multivariate Markov chains with polynomial eigenfunctions. Ann. Appl. Probab. 19 737–777.CrossRefGoogle Scholar
Koekoek, R. and Swarttouw, R. F. (1998) The Askey-scheme of hypergeometric orthogonal polynomials and its q-analogue. arXiv:9602214.Google Scholar
Kontoyiannis, I. and Meyn, S. P. (2003) Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13 304–362.Google Scholar
Kontoyiannis, I. and Meyn, S. P. (2005) Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes. Electron. J. Probab. 10 61–123.Google Scholar
Kontoyiannis, I. and Meyn, S. P. (2012) Geometric ergodicity and the spectral gap of non-reversible Markov chains. Probab. Theory Related Fields 154 327–339.CrossRefGoogle Scholar
Ledermann, W. and Reuter, G. E. H. (1954) Spectral theory for the differential equations of simple birth and death processes. Philos. Trans. Roy. Soc. London. Ser. A. 246 321–369.Google Scholar
Levin, D. A., Peres, Y. and Wilmer, E. L. (2009) Markov Chains and Mixing Times, American Mathematical Society.Google Scholar
Mao, Y. H., Zhang, C. and Zhang, Y. H. (2016) Separation cutoff for upward skip-free chains. J. Appl. Probab. 53 299–306.Google Scholar
Micchelli, C. A. and Willoughby, R. A. (1979) On functions which preserve the class of Stieltjes matrices. Linear Algebra Appl. 23 141–156.CrossRefGoogle Scholar
Miclo, L. (2010) On absorption times and Dirichlet eigenvalues. ESAIM Probab. Statist. 14 117–150.CrossRefGoogle Scholar
Miclo, L. (2015) An absorbing eigentime identity. Markov Process. Related Fields 21 249–262.Google Scholar
Miclo, L. (2019) On the Markov commutator. Bull. Sci. Math. 154 1–35.CrossRefGoogle Scholar
Miclo, L. (2018) On the Markovian similarity. Séminaire de Probabilités XLIX 375–403.Google Scholar
Montenegro, R. and Tetali, P. (2006) Mathematical aspects of mixing times in Markov chains. Found. Trends Theor. Comput. Sci. 1 237–354.CrossRefGoogle Scholar
Patie, P. and Savov, M. Spectral expansion of non-self-adjoint generalized Laguerre semigroups. To appear in Mem. Amer. Math. Soc.Google Scholar
Patie, P. and Zhao, Y. (2017) Spectral decomposition of fractional operators and a reflected stable semigroup. J. Diff. Equations 262 1690–1719.CrossRefGoogle Scholar
Paulin, D. (2015) Concentration inequalities for Markov chains by Marton couplings and spectral methods. Electron. J. Probab. 20 79.CrossRefGoogle Scholar
Peres, Y. (2004) American Institute of Mathematics (AIM) research workshop ‘Sharp Thresholds for Mixing Times’ (Palo Alto, December 2004). Summary available at http://www.aimath.org/WWN/mixingtimes.Google Scholar
Pike, J. (2013) Eigenfunctions for random walks on hyperplane arrangements. PhD thesis, University of Southern California.Google Scholar
Rosenthal, J. S. and Rosenthal, P. (2015) Spectral bounds for certain two-factor non-reversible MCMC algorithms. Electron. Commun. Probab. 20 91.CrossRefGoogle Scholar
Sasaki, R. (2009) Exactly solvable birth and death processes. J. Math. Phys. 50 103509.CrossRefGoogle Scholar
Schoutens, W. (2000) Stochastic Processes and Orthogonal Polynomials, Vol. 146 of Lecture Notes in Statistics, Springer.CrossRefGoogle Scholar
Siegmund, D. (1976) The equivalence of absorbing and reflecting barrier problems for stochastically monotone Markov processes. Ann. Probab. 4 914–924.CrossRefGoogle Scholar
Thompson, R. C. (1977) Singular values, diagonal elements, and convexity. SIAM J. Appl. Math. 32 39–63.CrossRefGoogle Scholar
Villani, C. (2009) Hypocoercivity, Vol. 202, no. 950, of Memoirs of the American Mathematical Society, American Mathematical Society.CrossRefGoogle Scholar
Wermer, J. (1954) Commuting spectral measures on Hilbert space. Pacific J. Math. 4 355–361.CrossRefGoogle Scholar
Young, R. M. (2001) An Introduction to Nonharmonic Fourier Series, first edition, Academic Press.Google Scholar
Zhou, H. (2008) Examples of multivariate Markov chains with orthogonal polynomial eigenfunctions. PhD thesis, Stanford University.Google Scholar