# Statisfaction

## New article about MCMC and parallel computation

Posted in Uncategorized by Pierre Jacob on 11 October 2010

Hi there,

Christian P. Robert, Murray Smith and I have just finished an article (arxiv link) on an improved Independent Metropolis-Hastings (IMH) algorithm that can highly benefit from parallel processing units (see this previous post for an introduction on parallel computation).

Our method named “block Independent Metropolis-Hastings” allows significant decrease of the variance of estimators based on the IMH, while being cost-free in a parallel processing context (and still cheap in a single core context, since you don’t do any additional target density evaluations compared to the standard IMH algorithm).

We applied our method on a toy example and a probit regression model on the classic “diabetes in Pima Indian Women” included in R. You can load the dataset with the following R commands:

```library(MASS)
data(Pima.te)
?Pima.te```

Update: my PhD advisor Christian P. Robert blogs about the article as well, in more details.

Update2: I’ve put the code online here (you need Python, numpy, scipy.weave for the computation part, plus R and rpy2 for the plots).

### 9 Responses

1. Julyan said, on 12 October 2010 at 16:52

Great!! Nice picture I remember well from Pierre’s blackboard in the PhD student seminar where we had the early benefit of the work…

2. Colin Gillespie said, on 13 October 2010 at 11:17

Hi Peirre,

Is it possible to obtain a copy of your R code that your used in the paper. Also, why is new approach superior to just running b independent chains?

Thanks

• pierrejacob said, on 13 October 2010 at 18:03

Hi Colin,

Thanks for your interest. The code is mostly in Python, the graphs are made using R indeed. I’ll clean and comment it, and put it online. I might update the blog during the day with a raw version of the it.

About your question, we tried to anticipate it in our conclusion (page 20). I’ll rephrase it here. Running p independent chains for T iterations costs T * p target density evaluations, so it is not comparable with our method which relies on T target density evaluations (whether you can compute them in parallel or not does not matter here). Conversely if you would like to use p independent chains, each of the chain can benefit from our method without computing any additional target density evaluations. So running p independent chains is not really an alternative method, it is just another thing, than can benefit from our approach as well.

• pierrejacob said, on 13 October 2010 at 19:24

Quick update: I’ve put the code online here:
http://dl.dropbox.com/u/1430652/blockIMH.zip
Hope this helps. You can test it with “python launch.py” in the extracted folder.
If it does not work instantly, check the README file.

• Colin Gillespie said, on 13 October 2010 at 19:27

Thanks. I’ll let you know how I get on.

3. […] whereas the standard Metropolis-Hastings algorithm is essentially sequential. See as well Pierre, Christian and Murray Smith’s block Independent Metropolis-Hastings algorithm for further […]

4. MCM’Ski / Adap’ski « Statisfaction said, on 6 December 2010 at 13:28

[…] will present a poster on an article I’ve talked about here, and available on arXiv here. It describes a method to make the most of each target density […]

5. […] the next days, I will be attending MCM’ski where I will present a poster based on our article on parallel post-processing of the IMH algorithm. This poster is available […]

6. […] October, I blogged about an article written by Christian P. Robert, Murray Smith and myself about parallel […]