In this paper we consider the optimal scaling of high-dimensional random walk Metropolis algorithms for densities differentiable in the L p mean but which may be irregular at some points (such as the Laplace density, for example) and/or supported on an interval. Our main result is the weak convergence of the Markov chain (appropriately rescaled in time and space) to a Langevin diffusion process as the dimension d goes to ∞. As the log-density might be nondifferentiable, the limiting diffusion could be singular. The scaling limit is established under assumptions which are much weaker than the one used in the original derivation of Roberts et al. (1997). This result has important practical implications for the use of random walk Metropolis algorithms in Bayesian frameworks based on sparsity inducing priors.

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

The possible existence of Internal Labour Markets (ILMs) in the Spanish railway company MZA at the beginning of the Second Industrial Revolution is explored by means of non-linear Optimal Scaling techniques. The results show the presence of two characteristics of ILMs predicted by the theory: the existence of «ports of entry» for the employees of the company at low levels of qualification, and the presence of long-term labour relations. Thus, in common with recent studies on UK railway companies, we conclude that the company MZA operated some kind of ILM before the start of the 20th Century.