New article about MCMC and parallel computation

Posted in Uncategorized by Pierre Jacob on 11 October 2010

Hi there,

Christian P. Robert, Murray Smith and I have just finished an article (arxiv link) on an improved Independent Metropolis-Hastings (IMH) algorithm that can highly benefit from parallel processing units (see this previous post for an introduction on parallel computation).

Our method named “block Independent Metropolis-Hastings” allows significant decrease of the variance of estimators based on the IMH, while being cost-free in a parallel processing context (and still cheap in a single core context, since you don’t do any additional target density evaluations compared to the standard IMH algorithm).

We applied our method on a toy example and a probit regression model on the classic “diabetes in Pima Indian Women” included in R. You can load the dataset with the following R commands:


Update: my PhD advisor Christian P. Robert blogs about the article as well, in more details.

Update2: I’ve put the code online here (you need Python, numpy, scipy.weave for the computation part, plus R and rpy2 for the plots).

9 Responses

Subscribe to comments with RSS.

  1. Julyan said, on 12 October 2010 at 16:52

    Great!! Nice picture I remember well from Pierre’s blackboard in the PhD student seminar where we had the early benefit of the work…

  2. Colin Gillespie said, on 13 October 2010 at 11:17

    Hi Peirre,

    Is it possible to obtain a copy of your R code that your used in the paper. Also, why is new approach superior to just running b independent chains?


    • pierrejacob said, on 13 October 2010 at 18:03

      Hi Colin,

      Thanks for your interest. The code is mostly in Python, the graphs are made using R indeed. I’ll clean and comment it, and put it online. I might update the blog during the day with a raw version of the it.

      About your question, we tried to anticipate it in our conclusion (page 20). I’ll rephrase it here. Running p independent chains for T iterations costs T * p target density evaluations, so it is not comparable with our method which relies on T target density evaluations (whether you can compute them in parallel or not does not matter here). Conversely if you would like to use p independent chains, each of the chain can benefit from our method without computing any additional target density evaluations. So running p independent chains is not really an alternative method, it is just another thing, than can benefit from our approach as well.

  3. […] whereas the standard Metropolis-Hastings algorithm is essentially sequential. See as well Pierre, Christian and Murray Smith’s block Independent Metropolis-Hastings algorithm for further […]

  4. MCM’Ski / Adap’ski « Statisfaction said, on 6 December 2010 at 13:28

    […] will present a poster on an article I’ve talked about here, and available on arXiv here. It describes a method to make the most of each target density […]

  5. […] the next days, I will be attending MCM’ski where I will present a poster based on our article on parallel post-processing of the IMH algorithm. This poster is available […]

  6. […] October, I blogged about an article written by Christian P. Robert, Murray Smith and myself about parallel […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: