Hostname: page-component-78c5997874-v9fdk Total loading time: 0 Render date: 2024-11-11T00:02:54.291Z Has data issue: false hasContentIssue false
  • English
  • Français

On explicit form of the stationary distributions for a class of bounded Markov chains

Published online by Cambridge University Press:  24 March 2016

S. McKinlay  and
K. Borovkov
S. McKinlay*
Affiliation:
Department of Mathematics and Statistics, University of Melbourne, Parkville, VIC 3010, Australia.
K. Borovkov
Affiliation:
Department of Mathematics and Statistics, University of Melbourne, Parkville, VIC 3010, Australia. Email address: borovkov@unimelb.edu.au
*
** Email address: s.mckinlay@ms.unimelb.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider a class of discrete-time Markov chains with state space [0, 1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then the length of the jump is chosen independently as a random proportion of the distance to the respective end point of the unit interval, the distributions of the proportions being fixed for each of the two directions. Chains of that kind were the subjects of a number of studies and are of interest for some applications. Under simple broad conditions, we establish the ergodicity of such Markov chains and then derive closed-form expressions for the stationary densities of the chains when the proportions are beta distributed with the first parameter equal to 1. Examples demonstrating the range of stationary distributions for processes described by this model are given, and an application to a robot coverage algorithm is discussed.

Keywords

Stationary distributionMarkov chainergodicitybeta distributiongive-and-take modelsemidegenerate kernelrandom search
Type
Research Papers
Information
Journal of Applied Probability , Volume 53 , Issue 1 , March 2016 , pp. 231 - 243
Copyright
Copyright © Applied Probability Trust 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1]Białkowski, M. and Wesołowski, J. (2002). Asymptotic behavior of some random splitting schemes. Prob. Math. Statist. 22, 181–191. Google Scholar
[2]Borovkov, A. A. (1998). Ergodicity and Stability of Stochastic Processes. John Wiley, Chichester. Google Scholar
[3]Borovkov, K. A. (1994). On simulation of random vectors with given densities in regions and on their boundaries. J. Appl. Prob. 31, 205–220. Google Scholar
[4]DeGroot, M. H. and Rao, M. M. (1963). Stochastic give-and-take. J. Math. Anal. Appl. 7, 489–498. Google Scholar
[5]Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41, 45–76. Google Scholar
[6]Golberg, M. A. (1978). Boundary and initial-value methods for solving Fredholm equations with semidegenerate kernels. J. Optimization Theory Appl. 24, 89–131. Google Scholar
[7]Iacus, S. M. and Negri, I. (2003). Estimating unobservable signal by Markovian noise induction: when noise helps in statistics! Statist. Methods Appl. 12, 153–167. Google Scholar
[8]Li, C. (1961). Human Genetics. McGraw-Hill, New York. Google Scholar
[9]Luus, R. and Jaakola, T. H. I. (1973). Optimization by direct search and systematic reduction of the size of search region. Amer. Inst. Chem. Eng. J. 19, 760–766. Google Scholar
[10]McKinlay, S. (2014). A characterisation of transient random walks on stochastic matrices with Dirichlet distributed limits. J. Appl. Prob. 51, 542–555. CrossRefGoogle Scholar
[11]Pacheco-González, C. G. (2009). Ergodicity of a bounded Markov chain with attractiveness towards the centre. Statist. Prob. Lett. 79, 2177–2181. Google Scholar
[12]Pacheco-Gonzalez, C. G. and Stoyanov, J. (2008). A class of Markov chains with beta ergodic distributions. Math. Scientist 33, 110–119. Google Scholar
[13]Ramli, M. A. and Leng, G. (2010). The stationary probability density of a class of bounded Markov processes. Adv. Appl. Prob. 42, 986–993. CrossRefGoogle Scholar
[14]Schrack, G. and Choit, M. (1976). Optimized relative step size random searches. Math. Programming 10, 230–244. CrossRefGoogle Scholar
[15]Spaans, R. and Luus, R. (1992). Importance of search-domain reduction in random optimization. J. Optimization Theory Appl. 75, 635–638. Google Scholar
[16]Stoyanov, J. and Pirinsky, C. (2000). Random motions, classes of ergodic Markov chains and beta distributions. Statist. Prob. Lett. 50, 293–304. CrossRefGoogle Scholar
[17]Zabinsky, Z. B. (2003). Stochastic Adaptive Search for Global Optimization. Kluwer, Boston, MA. CrossRefGoogle Scholar