Hostname: page-component-cd9895bd7-7cvxr Total loading time: 0 Render date: 2024-12-28T14:43:33.780Z Has data issue: false hasContentIssue false
  • English
  • Français

Does Waste Recycling Really Improve the Multi-Proposal Metropolis–Hastings algorithm? an Analysis Based on Control Variates

Part of: Limit theorems Probabilistic methods, simulation and stochastic differential equations Markov processes Equilibrium statistical mechanics

Published online by Cambridge University Press:  14 July 2016

Jean-Françcois Delmas  and
Benjamin Jourdain
Jean-Françcois Delmas*
Affiliation:
Université Paris-Est
Benjamin Jourdain*
Affiliation:
Université Paris-Est
*
∗Postal address: CERMICS, École des Ponts, 6–8 Avenue Blaise Pascal, Champs-sur-Marne, 77455 Marne La Vallée, France.
∗Postal address: CERMICS, École des Ponts, 6–8 Avenue Blaise Pascal, Champs-sur-Marne, 77455 Marne La Vallée, France.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The waste-recycling Monte Carlo (WRMC) algorithm introduced by physicists is a modification of the (multi-proposal) Metropolis–Hastings algorithm, which makes use of all the proposals in the empirical mean, whereas the standard (multi-proposal) Metropolis–Hastings algorithm uses only the accepted proposals. In this paper we extend the WRMC algorithm to a general control variate technique and exhibit the optimal choice of the control variate in terms of the asymptotic variance. We also give an example which shows that, in contradiction to the intuition of physicists, the WRMC algorithm can have an asymptotic variance larger than that of the Metropolis–Hastings algorithm. However, in the particular case of the Metropolis–Hastings algorithm called the Boltzmann algorithm, we prove that the WRMC algorithm is asymptotically better than the Metropolis–Hastings algorithm. This last property is also true for the multi-proposal Metropolis–Hastings algorithm. In this last framework we consider a linear parametric generalization of WRMC, and we propose an estimator of the explicit optimal parameter using the proposals.

Keywords

Metropolis–Hastings algorithmmulti-proposal algorithmMonte Carlo Markov chainvariance reductioncontrol variatesergodic theoremcentral limit theorem

MSC classification

Secondary: 60F05: Central limit and other weak theorems 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60J22: Computational methods in Markov chains 65C40: Computational Markov chains 82B80: Numerical methods (Monte Carlo, series resummation, etc.)
Type
Research Article
Information
Journal of Applied Probability , Volume 46 , Issue 4 , December 2009 , pp. 938 - 959
Copyright
Copyright © Applied Probability Trust 2009 

Footnotes

Research supported by the ANR program ADAP'MC.

References

[1] Andersen, H. C. and Diaconis, P. (2007). Hit and run as a unifying device. J. Soc. Fran. Stat. 148, 5–28.Google Scholar
[2] Atchadé, Y. F. and Perron, F. (2005). Improving on the independent Metropolis–Hastings algorithm. Statistica Sinica 15, 3–18.Google Scholar
[3] Athènes, M. (2007). Web ensemble averages for retrieving relevant information from rejected Monte Carlo moves. Europ. Phys. J. B 58, 83–95.Google Scholar
[4] Ceperley, D., Chester, G. V. and Kalos, M. H. (1977). Monte Carlo simulation of a many fermion study. Phys. Rev. B 16, 3081–3099.Google Scholar
[5] Douc, R. and Robert, C. P. (2009). A vanilla Rao–Blackwellisation of Metropolis–Hastings algorithms. Preprint. Available at http://arXiv.org/0904.2144.Google Scholar
[6] Duflo, M. (1997). Random Iterative Models (Appl. Math. 34). Springer, Berlin.Google Scholar
[7] Frenkel, D. (2004). Speed-up of Monte Carlo simulations by sampling of rejected states. Proc. Nat. Acad. Sci. USA 101, 17571–17575.CrossRefGoogle ScholarPubMed
[8] Frenkel, D. (2006). Waste-recycling Monte Carlo. In Computer Simulations in Condensed Matter: From Materials to Chemical Biology (Lecture Notes Phys. 703), Springer, Berlin, pp. 127–137.Google Scholar
[9] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.Google Scholar
[10] Munos, R. (2006). Geometric variance reduction in Markov chains: application to value function and gradient estimation. J. Mach. Learn. Res. 7, 413–427.Google Scholar
[11] Peskun, P. H. (1973). Optimum Monte-Carlo sampling using Markov chains. Biometrika 60, 607–612.Google Scholar
[12] Robert, C. P. and Casella, G. (1999). Monte Carlo Statistical Methods. Springer, New York.Google Scholar