Dirichlet process and related priors
My contribution this year to MCB seminar at CREST is about nonparametric Bayes (today at 2 pm, room 14). I shall start with 1) a few words of history, then introduce 2) the Dirichlet Process by several of its numerous defining properties. I will next introduce an extension of the Dirichlet Process, namely 3) the DP mixtures, useful for applications like 4) density estimation. Last, I will show 5) posterior MCMC simulations for a density model and give some 6) reference textbooks.
Ferguson, T.S. (1973), A Bayesian Analysis of Some Nonparametric Problems. The Annals of Statistics, 1, 209-230.
Doksum, K. (1974). Tailfree and neutral random probabilities and their posterior distributions. The Annals of Probability, 2 183-201.
Blackwell, D. (1973). Discreteness of Ferguson selections. The Annals of Statistics, 1 356-358.
Blackwell, D. and MacQueen, J. (1973). Ferguson distributions via Polya urn schemes. The Annals of Statistics, 1 353-355.
2) Dirichlet Process defining properties
Mainly based on Peter Müller’s slides of class 2 at Santa Cruz this summer.
Dirichlet distribution on finite partitions
Stick-breaking/ Sethuraman representation
Polya urn analogy for the predictive probability function
Normalization of a Gamma Process
3) Dirichlet Process Mixtures (DPM)
Convolution of a DP with a continuous kernel to circumvent its discretness.
4) Density estimation with DPMs
5) Posterior MCMC simulation
Based on Peter Müller’s slides of class 6.
Bayesian nonparametrics, 2003, J. K. Ghosh, R. V. Ramamoorthi.
Bayesian nonparametrics, 2010, Nils Lid Hjort, Chris Holmes, Peter Müller, Stephen G. Walker, and contributions by Subhashis Ghosal, Antonio Lijoi, Igor Prünster, Yee Whye Teh, Michael I. Jordan, Jim Griffin, David B. Dunson, Fernando Quintana.