Handbook of Markov Chain Monte Carlo

Edited by Steve Brooks, Andrew Gelman, Galin L. Jones and Xiao-Li Meng

Published by Chapman & Hall/CRC.

Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisheries science and economics. The wide-ranging practical importance of MCMC has sparked an expansive and deep investigation into fundamental Markov chain theory.

The Handbook of Markov Chain Monte Carlo provides a reference for the broad audience of developers and users of MCMC methodology interested in keeping up with cutting-edge theory and applications. The first half of the book covers MCMC foundations, methodology and algorithms. The second half considers the use of MCMC in a variety of practical applications including some in educational research, astrophysics, brain imaging, ecology and sociology.

The in-depth introductory section of the book allows graduate students and practicing scientists new to MCMC to become thoroughly acquainted with the basic theory, algorithms and applications. The book supplies detailed examples and case studies of realistic scientific problems presenting the diversity of methods used by the wide-ranging MCMC community. Those familiar with MCMC methods will find this book a useful refresher of current theory and recent developments.


Markov chain Monte Carlo (MCMC) is a family of algorithms used to produce approximate random samples from a probability distribution too difficult to sample directly. The method produces a Markov chain that whose equilibrium distribution matches that of the desired probability distribution. Since Markov chains are typically not independent, the theory of MCMC is more complex than that of simple Monte Carlo sampling.

MCMC was invented in 1953 but was not widely applied until statisticians became aware of it around 1990. Then problems that had previously been intractable suddenly became tractable. MCMC worked like magic. (We will return to the comparison with magic below.)

One of the most popular books on MCMC to date has been Markov Chain Monte Carlo in Practice. This book was edited by Gilks, Richardson, and Spiegelhalter and published by Chapman & Hall in 1995, just five years after the use of MCMC became wide-spread among statisticians. Now Chapman & Hall has published a new book Handbook of Markov Chain Monte Carlo, edited by Brooks, Gelman, Jones, and Ming. The Handbook is in some sense an update to MCMC in Practice reflecting 16 years of theoretical development and experience using MCMC.

After two decades of widespread use, MCMC remains magical, both in the sense of being powerful and in the sense of being mysterious. As Charles Geyer explains in the book’s opening chapter:

"There is a great deal of theory about the convergence of Markov chains. Unfortunately, none of it can be applied to get useful convergence information for most MCMC applications."

Sometimes the magic can backfire in the form of pseudo-convergence. A Markov chain can appear to have converged when in fact it has not. Again Geyer explains:

"People who have used MCMC in complicated problems can tell stories about samples that appeared to be converging until, after weeks of running, they discovered a new part of the state space and the distribution changed radically."

Despite the element of mystery in MCMC, no practical computational alternative exists for many problems. Fortunately, MCMC often works better in practice than we have reason to expect by theory. Because there is no definitive theory to settle some of the practical issues in applying MCMC, there is much room for differing preferences. The Handbook has 42 contributors and four editors; differing approaches are inevitable. For example, Andrew Gelman and Kenneth Shirley advocate monitoring convergence using multiple Markov chains in one chapter while Charles Geyer argues against this approach in another chapter.

The Handbook consists of 24 chapters, arranged into two parts, the first entitled “Foundations, Methodology, and Algorithms.” and the second “Applications and Case Studies.” The line between algorithms and applications is somewhat blurry and so some of the chapters in the first part lean more toward application than the title might lead one to expect. The first part also contains valuable historical information. This history helps identify which contemporary practices are based more on tradition than demonstrated advantage. The second part gives examples of MCMC algorithms applied to a wide variety of areas: ecology, medical imaging, social science, etc.

Handbook of Markov Chain Monte Carlo is a valuable resource for those new to MCMC as well as to experienced practitioners. It is not a handbook analogous to, say, the Boy Scout Handbook. The field of MCMC is not settled enough for such a handbook to be possible. Instead, it is a collection of valuable information regarding a powerful computational approach to evaluating complex statistical models.

Reviewed by John D. Cook for MAA, on 25th June 2011


The editors of Handbook of Markov Chain Monte Carlo (Handbook, hereafter) start the second paragraph of their preface by stating that “this book is intended to be a reference (not a text) for a broad audience and to be used both to developers and users of MCMC methodology,” which is a statement that I vouch 100 percent! Nonetheless, the uninitiated would greatly benefit from carefully reading the first three chapters by Geyer (Chapter 1), Robert and Casella (Chapter 2), and Fan and Sisson (Chapter 3), preferably in this order. For a more experienced and hands-on reader who is looking for up-to-date developments and implementations of cutting-edge MCMC tools for highly complex problems, the Handbook brings twelve chapters with applications and case studies from areas ranging from genetics to high-energy astrophysics to item response theory to fisheries science.

Section 1.1 and most of Chapter 2 provide a nice helicopter tour over the historical trajectory of Monte Carlo schemes, from its origins in chemistry and physics in the 1950’s to spatial statistics and image analysis in the 1970’s and 1980’s to its almost instantaneous celebrity status within the Bayesian community after the publication of Gelfand and Smith (1990). Chapter 2, by Robert and Casella, splits the history of MCMC into three parts: before, during and after the Revolution. The work by Metropolis et al. (1953) and Hastings (1970), and their contemporaries, fall into the before the Revolution part. The Revolution period comprises Besag’s image analysis quest as well as Geman and Geman’s statistical physics problems (see also Haran’s Chapter 18, Section 18.1.2 on MCMC and Spatial Models: A Shared History). Particle systems, perfect sampling and reversible jump MCMC are among the post Revolution schemes. In fact, one of the most popular MCMC schemes is the reversible jump MCMC. Chapter 3 is a great contribution from Fan and Sisson on this subject by listing a number of areas of application of reversible jump MCMC: change-point and mixture models, variable selection, nonparametrics and time series models.

The Practice of MCMC (Section 1.11) is a “must-read” section mainly because it argues that it is rather difficult, sometimes impossible, to answer the questions you instantly ask yourself (or your colleague, instructor, advisor) when implementing an MCMC algorithm for the first time, say a Gibbs sampler for the simple linear regression with nonconjugate priors: one long run or many short runs, burn-in, and what convergence diagnostic should I use? For those who want more discussion, Chapter 6 by Gelman and Shirley provides a quick summary of recommendations (Section 6.1) and more discussion about burn-in and convergence diagnostics.

The vast majority of the chapters on applications and case studies (Chapters 13–24) should be of interest to most readers. Nonetheless, different groups of applications might help specific audiences mature on the potentials as well as limitations of MCMC schemes. For example,

(1) Chapters 13, 14, and 20 might be suitable as a first application chapter for those readers in the fields of genetics, bioinformatics, biostatistics and epidemiology.

(2) Readers in the fields of astrophysics, computer modelling and spatial statistics by benefit from reading Chapters 15, 16, and 18.

(3) Social scientists, in particular policy makers and evaluators, and sociologists might relate to Chapters 19, 22, and 24.

(4) Chapters 17 and 23 deal with statistical ecology problems and fisheries science.

(5) Unfortunately, Chapter 20, on state-space models, provides a rather limited account to a very important class of time series models and, perhaps more seriously, cites virtually none of the well-established contributions to the area.

Several MCMC schemes are adapted, extended, or proposed in the case study chapters, such as partially collapsed Gibbs sampler and path-adaptive Metropolis–Hastings sampler (Chapter 15), block Gibbs sampler for hidden Markov models (Chapter 13), differential evolution MCMC and delayed acceptance Metropolis (Chapter 16), amongst others.

Going back to part I of the Handbook, which comprises the first 12 chapters. Perhaps Chapter 4, by Jeff Rosenthal, should be included above along with Chapters 1–3, since it deals with automatic scale-tuning of random walk Metropolis–Hastings proposals as well as other adaptive MCMC schemes. But again, I think I will keep it here so the novice reader will first try running a few old-school MCMC schemes and finding herself into well-known troubles (well-known to us, of course!). Another couple of chapters that fall into this category are Flegal and Jones’ Chapter 7 and, once the reader is more mature, Geyer’s Chapter 11 on importance sampling, simulated tempering and umbrella sampling. Finally, the interesting (and controversial) Chapter 12 on likelihood-free (aka approximate Bayesian computation) address the issue of Bayesian analysis for models which are sufficiently complex that the likelihood function is either analytically intractable or computationally prohibitive to evaluate, using Sisson and Fan’s own words.

To sum up, it is my opinion that the Handbook might play, for the next decade or so at least, more or less the same role played by MCMC in Practice in the mid 90’s. More precisely, the Handbook is bounded to provide theoretical and practical guidance to Bayesian researchers and practitioners (and why not non-Bayesians alike) to design, implement (and debug) and package their own MCMC routines by taking advantage of the various inexpensive computational tools and the pressing need to analyze larger and more complex data frames. Or, as Higdon et al. say at the very end of the discussion of their Chapter 16, “The integration of modern computing architecture with MCMC methods will certainly extend the reach of MCMC based solutions to inverse problems.” You can replace inverse problems“ simply by “complex models.”


Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B 36, 192–236.

Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association 85, 398–409.

Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741.

Gilks, W. R., Richardson, S., and Spiegelhalter, D. (1995). Markov Chain Monte Carlo in Practice. Chapman & Hall/CRC Interdisciplinary Statistics, Boca Raton, USA.

Hastings, W.K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109.

Metropolis, N. (1953). Equation of state calculations by fast computing machines. The Journal of Chemical Physics 21, 1087–1092.

Reviewed by Hedibert Freitas Lopes for Biometrics, October 2013

Valid HTML 4.01 Strict

Valid CSS!