Dirichlet process and related priors

Posted in Seminar/Conference by Julyan Arbel on 23 September 2010

Johann Dirichlet

My contribution this year to MCB seminar at CREST is about nonparametric Bayes (today at 2 pm, room 14). I shall start with 1) a few words of history, then introduce 2) the Dirichlet Process by several of its numerous defining properties. I will next introduce an extension of the Dirichlet Process, namely 3) the DP mixtures, useful for applications like 4) density estimation. Last, I will show 5) posterior MCMC simulations for a density model and give some 6) reference textbooks.

1) History

Kjell Doksum

Ferguson, T.S. (1973), A Bayesian Analysis of Some Nonparametric Problems. The Annals of Statistics, 1, 209-230.
Doksum, K. (1974). Tailfree and neutral random probabilities and their posterior distributions. The Annals of Probability, 2 183-201.
Blackwell, D. (1973). Discreteness of Ferguson selections. The Annals of Statistics, 1 356-358.
Blackwell, D. and MacQueen, J. (1973). Ferguson distributions via Polya urn schemes. The Annals of Statistics, 1 353-355.

2) Dirichlet Process defining properties

Mainly based on Peter Müller’s slides of class 2 at Santa Cruz this summer.

Dirichlet distribution on finite partitions
Stick-breaking/ Sethuraman representation
Polya urn analogy for the predictive probability function
Normalization of a Gamma Process

3) Dirichlet Process Mixtures (DPM)

Convolution of a DP with a continuous kernel to circumvent its discretness.

4) Density estimation with DPMs

5) Posterior MCMC simulation

Based on Peter Müller’s slides of class 6.

6) Textbooks

Bayesian nonparametrics, 2003, J. K. Ghosh, R. V. Ramamoorthi.
Bayesian nonparametrics, 2010, Nils Lid Hjort, Chris Holmes, Peter Müller, Stephen G. Walker, and contributions by Subhashis Ghosal, Antonio Lijoi, Igor Prünster, Yee Whye Teh, Michael I. Jordan, Jim Griffin, David B. Dunson, Fernando Quintana.


4 Responses

Subscribe to comments with RSS.

  1. Paul said, on 5 October 2010 at 05:23

    The first chapter of Schervish’s “Theory of Statistics” contains excelent materials on the Dirichlet Process.

  2. Julyan said, on 8 October 2010 at 11:32

    Thank you Paul for this reference. The Lemma on the marginal distribution of observations from a DP is particularly useful!

  3. […] mentionned in an earlier post, a foundational paper and still a nice reading today, which introduced the DP, is Ferguson, T.S. […]

  4. […] work on Projective limit random probabilities on Polish spaces. It will follow my previous presentation about Bayesian nonparametrics on the Dirichlet […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: