Back from ISBA Regional Meeting in India

Hello everyone,

and of course Happy New Year (2013 is the international year of statistics!).

Last week the ISBA Regional Meeting was held in Banaras / Varanasi, in the North of India. The conference was well attended, with leading figures such as Jayanta K. Ghosh, José Bernardo, James Berger, Peter Green, Christian Robert who blogged about it, and an overall ~350 participants.

A glimps of Inverse Problems

Hi folks !

Last Tuesday a seminar on Bayesian procedure for inverse problems took place at CREST. We had time for two presentations of young researchers Bartek Knapik and Kolyan Ray. Both presentations deal with the problem of observing a noisy version of a linear transform of the parameter of interest

where is a linear operator and a Gaussian white noise.  Both presentations considered asymptotic properties of the posterior distribution (Their papers can be found on arxiv, here for Bartek’s, and here for Kolyan’s). There is a wide literature on asymptotic properties of the posterior distribution in direc models. When looking at the concentration of  toward a true distribution   given the data, with respect to some distance ,  well known problem is to derive concentration rates, that is the rate such that

For inverse problems, the usual methods as introduced by Ghosal, Ghosh and van der Vaart (2000) usually fails, and thus results in this settings are in general difficult to obtain.

Bartek presented some very refined results in the conjugate case. He manages to get some results on the concentration rates of the posterior distribution, on Bayesian Credible Sets and Bernstein – Von Mises theorems – that states that the posterior is asymptotically Gaussian – when estimating a linear functional of the parameter of interest. Kolyan got some general conditions on the prior to achieve concentration rate, and prove that these techniques leads to optimal concentration rates for classical models.

I only knew little about inverse problems but both talks were very accessible and I will surely get more involved in this field !

Recent Advances in Sequential Monte Carlo / Warwick 2012

Hello,

This blog is not dead! And it’s gonna get more active soon.

These last few days, a workshop on Sequential Monte Carlo methods was held in the University of Warwick (link to the webpage). It was a very exciting meeting, efficiently organised by Arnaud Doucet, Adam Johansen, Anthony Lee and Murray Pollock and hosted by CRiSM. For those who couldn’t attend, here’s a little summary of my experience (or more exactly, just a bunch of links). Since SMC methods are at the core of my research, I was logically interested by all the talks (which is exceptional for 3 days of workshop, filled with 30 talks!). It was probably a good time for a workshop on SMC, since there’s a lot of recent activity in the field. My impression is that this renewed interest is mainly due to:

• the Particle MCMC framework, which inspired other algorithms (like our SMC^2 that Nicolas Chopin presented, or the Particle Gibbs with ancestral sampling), renewed interest in the normalising constant estimate in particle filters, and more generally made it possible to estimate parameters in a broad class of time series models with plenty of diverse applications (probably more than 200 citations already);
• theoretical advances, with recent papers from Pierre Del Moral, Nick Whiteley, Ajay Jasra, and many others;
• a new class of exact algorithms for simulating continuous time processes without any discretisation error, based on sequential importance sampling, in a fascinating work by Paul Fearnhead, Krzysztof Łatuszyński and colleagues.
• parallel computing, including the recent GPU trend, which makes SMC all the more attractive compared to purely iterative algorithms like MCMC.

The last point was illustrated at this workshop by a recent work from Alexandre Bouchard-Côté and colleagues called “Entangled Monte Carlo”, as well as by my own presentation: I talked about a new resampling scheme that avoids global interactions between all the particles, and resorts only to multiple pair-wise interactions. This is an on-going work with Pierre Del Moral, Anthony Lee, Lawrence Murray and Gareth Peters, that I might talk about again with more details in the future!

Cheers!

Priors on probability measures

Hi,

for the next GTB meeting at Crest, 3rd May, I will present Peter Orbanz‘ work on Projective limit random probabilities on Polish spaces. It will follow my previous presentation about Bayesian nonparametrics on the Dirichlet process.

The article provides a means of constructing any arbitrary prior distribution on the set of probability measures by working on its finite-dimensional marginals. The vanilla example is the Dirichlet process, which is characterized by its Dirichlet distribution marginals on any finite partition of the space (other examples are the Normalized Inverse Gaussian Process and the Pòlya Tree). The figure above illustrates the projective property of the marginals.

Peter will speak at ISBA 2012 Kyoto session : On the uses of random probabilities in Bayesian inference, along with Ramses Mena and Antonio Lijoi. I’ll write more about that later on!

Awesome Bristol

Hey,

Last week there was a workshop on Confronting Intractability in Statistical Inference, organised by the University of Bristol and the SuSTain group. It was hosted at the Goldney Hall (picture above). It turned out to be a succession of fascinating talks about the recent developments and the future of statistical methods used in very challenging inference problems. What I appreciated above all was the ambition of many talks, and the generosity of the speakers in giving many ideas to the audience.

Among the things I’ve learned there, the following were the most ambitious in my opinion.

Rochebrune Workshop 2012

Hey,
Last week I attended Rochebrune workshop for the second time. The genius organizers’ idea (Liliane Bel and Eric Parent from AgroParisTech, Jean-Jacques Borreux from Liège University) is to mix ski, stats and spirits (mostly Genepi and Chartreuse) around a remote alpine chalet on top of Megève ski resort.

Most of the attendees are (young) Bayesians working in applied fields, ranging from biology, ecology and epidemiology, to meteorology and climatology. We had great talks about fishes, trees, birds (Joël’s busard cendré), drugs and avalanches. More methodological talks dealt with extremes, Bayesian model averaging, and simulations: variational approximations, INLA, ABC, and MCMC in general. We had a tutorial about JAGS and WinBugs/OpenBugs as well (and how to interface them with R using rjags and R2WinBUGS). I presented my work about Multidimensional covariate dependent Dirichlet processes (all presentations here).

In addition to the 10 talks per day, a sacrosanct 5-hour skiing slot was reserved in the afternoon, with lessons from crazy Mégevan instructors. They must be really good: Pierre, don’t be afraid, I jumped and felt significantly less than two years ago. Have a Chartreuse, cheers!

Next stop: Sydney

Hey hey,

After a very interesting week in the Department of Statistics and Applied Probability at the National University of Singapore, where I’ve started some projects about the mathematics behind Sequential Monte Carlo methods with the expert Ajay Jasra, I’m heading towards Sydney, Australia. I love my life at the moment!

Tip for the travelers among you: ScribTeX is a very convenient website if you travel without a laptop. You can type LaTeX documents online and also compile them, and see the resulting PDF. They seem to have all the necessary packages and I could easily type beamer presentations. The editor is a bit simple but it could still save your life!

Speaking of slides, I will be talking next Thursday at the MCQMC conference. My office mate Christian Schaefer will be there too if you wish to speak to an expert in SMC on binary spaces. As for me, I’ll be talking about the properties of the stochastic schedule in the Wang-Landau algorithm, linked with our recent article with Robin Ryder.

Then on February 20th, I’ll give an informal talk at the Centre for Language Science in Macquarie University. That’s quite random, yes. Many thanks to Mark Johnson who contacted me through the blog! There I’ll speak about recent advances in computational Bayesian statistics. These linguists already use SMC for word segmentation and show interest in the latest advances. I’ll hence talk about PMCMC/SMC^2 and also ABC.

Bye bye.

Coming R meetings in Paris

If you live in Paris and are interested in R, there will be two meetings for you this week.

First a Semin-R session, organized at the Muséum National d’Histoire Naturelle on Tuesday 7 Feb (too bad, the Museum is closed on Tuesdays). Presentations will be about colors, phylogenies and maps, while I will speak about (my beloved) RStudio. The slides of previous sessions can be found here (most of them are in French).

The following day, 8 Feb, a group of R users from INSEE will have its first meeting (13-14h, INSEE, room R12), about SQLite data in R, maps, and in R.

I guess anyone can join!

UPDATE: Here is a colorful map to access INSEE . Come with an ID, and say you are visiting the meeting organizer Matthieu Cornec. Room R12 is on the ground floor (left).

Heading to the National University of Singapore

Hey there,

Next Wednesday (Feb 8th) I’ll give a talk on SMC² at the National University of Singapore (3pm, S16-06-118, DSAP Seminar Room if you’re interested).

By the way, the SMC² paper has been hugely revised, along with the python package and the supplement. Among the main changes, we made a thorough comparison with Liu & West SMC method (which is based on extending the hidden state to include the parameter), and we added a fair bit of justification of the computational cost of the method, which we believe is very reasonable considering the difficulty of the problem (ie estimation in HMM) and the cost of concurrent algorithms. We also considered the validity of SMC² on a sequence of annealed posterior densities, following a reviewer’s interesting comment. It’s good to be finished with it (for now at least), and to move to new projects!

I’ll spend next week in Singapore working with Ajay Jasra, and then I’ll head off to the Monte Carlo and Quasi Monte Carlo conference in Sydney, where I’ll stay for ~2 weeks. I’ll be worth another blog post of course.

Cheers!

GPUs in Computational Statistics

Hey,

Next week the Centre for Research in Statistical Methodology (CRiSM, in Warwick, UK) will be hosting a workshop on the use of graphics processing units in statistics, a quickly expanding area that I’ve blogged about here. Xian and I are going to talk about Parallel IMH and Parallel Wang Landau. We’ll be able to interact with top researchers in methodological statistics and early adopters of GPUs like Chris Holmes, whose talk at Valencia 9 was quite influential in the field (and for my PhD!), and Christophe Andrieu, who is one of the to-be-praised Particle MCMC guys (see e.g. here)

The programme for the workshop is here: http://www2.warwick.ac.uk/fac/sci/statistics/crism/workshops/gpu/programme

…and registration fee is only 50£, ie not even a half GPU!

By the way this page

http://www.louisaslett.com/Talks/GPU_Programming_Basics_Getting_Started.html

should help you a lot if you want to use plain CUDA C code within R.

As for me, thanks to Anthony Lee I will spend a week in Warwick prior to the meeting, working at CRiSM and enjoying the West Midlands.

Cheers!