Reading Bayesian classics — presentations

Posted in General by Julyan Arbel on 21 April 2015

The students did a great job in presenting some Bayesian classics. I enjoyed reading the papers (pdfs can be found here), most of which I hadn’t read before, and enjoyed also the students’ talks. I share here some of the best ones, as well as some demonstrative excerpts from the papers. In chronological order (presentations on slideshare below):

  • W. Keith Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1):97–109, 1970.

In this paper, we shall consider Markov chain methods of sampling that are generalizations of a method proposed by Metropolis et al. (1953), which has been used extensively for numerical problems in statistical mechanics.

  • Dennis V. Lindley and Adrian F.M. Smith. Bayes estimates for the linear model. Journal of the Royal Statistical Society: Series B (Statistical Methodology), with discussion, 1–41, 1972.

From Prof. B. de Finetti discussion (note the valliant collaborator Smith!):

I think that the main point to stress about this interesting and important paper is its significance for the philosophical questions underlying the acceptance of the Bayesian standpoint as the true foundation for inductive reasoning, and in particular for statistical inference. So far as I can remember, the present paper is the first to emphasize the role of the Bayesian standpoint as a logical framework for the analysis of intricate statistical situation. […] I would like to express my warmest congratulations to my friend Lindley and his valiant collaborator Smith.

  • Persi Diaconis and Donald Ylvisaker. Quantifying prior opinion. Bayesian Statistics, 2:133–156, 1985.

About the binomial model of spinning a coin:

Let us distinguish three categories of Bayesians (certainly a crude distinction in light of Good’s (1971) 46,656 lower bound on the possible types of Bayesians).

  1. Classical Bayesians. (Like Bayes, Laplace and Gauss) took f(p) \equiv 1. A so called flat prior.
  2. Modern Parametric Bayesians. (Raifa, Lindley, Mosteller) took f(p) as a beta density […]
  3. Subjective Bayesians. (Ramsey, de Finetti, Savage) take the prior as a quantification of what is known about the coin and spinning process.
  • Eric L. Lehmann. Model specification: the views of Fisher and Neyman, and later developments. Statistical Science, 5(2):160–168, 1990.

Where do probability models come from? To judge by the resounding silence over this question on the part of most statisticians, it seems highly embarrassing. In general, the theoretician is happy to accept taht his abstract probability triple (\boldsymbol{\Omega, A, P}) was found under a gooseberry bush, while the applied statistician’s model “just growed”. (quoting A. P. Dawid, 1982)

  • William H. Jefferys and James O. Berger. Ockham’s razor and Bayesian analysis. American Scientist, 64–72, 1992.

William of Ockham, the 14th-century English philosopher, stated the principle thus: “Pluralitas non est ponenda sine necessitate”, which can be translated as: “Plurality must not be posited without necessity.” […] Ironically, whereas Bayesian methods have been criticized for introducing subjectivity into statistical analysis, the Bayesian approach can turn Ockham’s razor into a less subjective and even “automatic” rule of inference.

  • Eugene Seneta. Lewis Carroll’s “pillow problems”: on the 1993 centenary. Statistical science, 180–186, 1993.

All 72 [Pillow Problems] are claimed to have been formulated and worked out at night while in bed, mentally, and the answer written down afterward. [C. L. Dodgson, a.k.a. Lewis Carroll] work reflects the nature, standing and understanding of probability within the wider English mathematical community of the time.

  • James O. Berger. Bayesian analysis: A look at today and thoughts of tomorrow. Journal of the American Statistical Association, 95(452):1269– 1276, 2000.

Life was simple when I became a Bayesian in the 1970s; it was possible to track virtually all Bayesian activity. Preparing this paper on Bayesian statistics was humbling, as I realized that I have lately been aware of only about 10% of the ongoing activity in Bayesian analysis.

Below, the seven presentations.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: