Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-27T07:28:49.109Z Has data issue: false hasContentIssue false
  • English
  • Français

Large deviations and full Edgeworth expansions for finite Markov chains with applications to the analysis of genomic sequences

Published online by Cambridge University Press:  22 December 2010

Pierre Pudlo
Pierre Pudlo*
Affiliation:
I3M, Université Montpellier 2, Place E. Bataillon, 34095 Montpellier, Cedex, France; pierre.pudlo@univ-montp2.fr
Get access

Abstract

To establish lists of words with unexpected frequencies in long sequences,for instance in a molecular biology context, one needs to quantify the exceptionality of families of word frequencies in random sequences. To this aim, we study large deviation probabilities of multidimensional word counts for Markov and hidden Markov models.More specifically, we compute local Edgeworth expansions of arbitrary degrees for multivariate partial sums of lattice valued functionals of finite Markov chains. This yields sharp approximations of the associated large deviation probabilities. We also provide detailed simulations. These exhibit in particular previously unreported periodic oscillations, for which we providetheoretical explanations.

Keywords

Markov chainshidden Markov modelslarge deviationsedgeworth expansionsprotein and DNA sequences
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 14 , 2010 , pp. 435 - 455
Copyright
© EDP Sciences, SMAI, 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Andriani, C. and Baldi, P., Sharp estimates of deviations of the sample mean in many dimensions. Ann. Inst. H. Poincaré Probab. Statist. 33 (1997) 371–385. CrossRef
Bahadur, R.R. and Rao, R.R., On deviations of the sample mean. Ann. Math. Statist. 31 (1960) 1015–1027. CrossRef
P. Barbe and M. Broniatowski, Large-deviation probability and the local dimension of sets, in Proceedings of the 19th Seminar on Stability Problems for Stochastic Models, Vologda, 1998, Part I. (2000), Vol. 99, pp. 1225–1233.
Chaganty, N.R. and Sethuraman, J., Strong large deviation and local limit theorems. Ann. Probab. 21 (1993) 1671–1690. CrossRef
Datta, S. and McCormick, W.P., On the first-order Edgeworth expansion for a Markov chain. J. Multivariate Anal. 44 (1993) 345–359. CrossRef
A. Dembo and O. Zeitouni, Large deviations techniques and applications. Volume 38 of Appl. Math. (New York). Second edition. Springer-Verlag, New York (1998).
Flajolet, P., Szpankowski, W. and Vallée, B., Hidden word statistics. J. ACM 53 (2006) 147–183 (electronic). CrossRef
Iltis, M., Sharp asymptotics of large deviations in R d. J. Theoret. Probab. 8 (1995) 501–522. CrossRef
Iltis, M., Sharp asymptotics of large deviations for general state-space Markov-additive chains in R d. Statist. Probab. Lett. 47 (2000) 365–380. CrossRef
Iscoe, I., Ney, P. and Nummelin, E., Large deviations of uniformly recurrent Markov additive processes. Adv. Appl. Math. 6 (1985) 373–412. CrossRef
J.L. Jensen, Saddlepoint approximations. The Clarendon Press Oxford University Press, New York (1995).
Kargin, V., A large deviation inequality for vector functions on finite reversible Markov chains. Ann. Appl. Probab. 17 (2007) 1202–1221. CrossRef
K. Knopp, Theory of Functions, Part I. Elements of the General Theory of Analytic Functions. Dover Publications, New York (1945).
Kontoyiannis, I. and Meyn, S.P., Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13 (2003) 304–362.
León, C.A. and Perron, F., Optimal Hoeffding bounds for discrete reversible Markov chains. Ann. Appl. Probab. 14 (2004) 958–970.
Lladser, M.E., Betterton, M.D. and Knight, R., Multiple pattern matching: a Markov chain approach. J. Math. Biol. 56 (2008) 51–92. CrossRef
B. Mann, Berry-Esseen Central Limit Theorems For Markov Chains. Ph.D. thesis, Harvard University, 1996.
Miller, H.D., A convexivity property in the theory of random variables defined on a finite Markov chain. Ann. Math. Statist. 32 (1961) 1260–1270. CrossRef
Dominating, P. Ney points and the asymptotics of large deviations for random walk on R d. Ann. Probab. 11 (1983) 158–167.
Ney, P. and Nummelin, E., Markov additive processes, Part I. Eigenvalue properties and limit theorems. Ann. Probab. 15 (1987) 561–592. CrossRef
P. Nicodème, B. Salvy and P. Flajolet, Motif statistics. In Algorithms – ESA '99, Prague. Lect. Notes Comput. Sci. 1643. Springer, Berlin (1999), pp 194–211.
G. Nuel, Numerical solutins for Patterns Statistics on Markov chains. Stat. Appl. Genet. Mol. Biol. 5 (2006).
Nuel, G., Pattern Markov chains: optimal Markov chain embedding through deterministic finite automata. J. Appl. Probab. 45 (2008) 226–243. CrossRef
R Development Core Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2003). ISBN 3-900051-00-3.
Régnier, M., A unified approach to word occurrence probabilities. Discrete Appl. Math. 104 (2000) 259–280, Combinatorial molecular biology. CrossRef
Régnier, M. and Denise, A., Rare events and conditional events on random strings. Discrete Math. Theor. Comput. Sci. 6 (2004) 191–213 (electronic).
Régnier, M. and Szpankowski, W., On pattern frequency occurrences in a Markovian sequence. Algorithmica 22 (1998) 631–649. CrossRef
G. Reinert, S. Schbath and M.S. Waterman, Applied Combinatorics on Words. In Encyclopedia of Mathematics and its Applications, Vol. 105, chap. Statistics on Words with Applications to Biological Sequences. Cambridge University Press (2005).
Robin, S. and Daudin, J.-J., Exact distribution of word occurrences in a random sequence of letters. J. Appl. Probab. 36 (1999) 179–193. CrossRef
Roquain, E. and Schbath, S., Improved compound Poisson approximation for the number of occurrences of any rare word family in a stationary Markov chain. Adv. Appl. Probab. 39 (2007) 128–140.
Schbath, S., Compound Poisson approximation of word counts in DNA sequences. ESAIM: PS 1 (1997) 1–16. CrossRef
D. Serre, Matrices, volume 216 of Graduate Texts Math.. Springer-Verlag, New York (2002). Theory and applications, translated from the 2001 French original.
Stefanov, V.T., Robin, S. and Schbath, S., Waiting times for clumps of patterns and for structured motifs in random sequences. Discrete Appl. Math. 155 (2007) 868–880. CrossRef