What is the best introductory Bayesian statistics textbook?
Which is the best introductory textbook for Bayesian statistics?
One book per answer, please.
In the replies, **please explain _why_ you are recommending a book** as "the best."
How can there be more than one answer to a question question phrased like this?
This is an old thread now, but I came back to +1 a new book "Statistical Rethinking. And in looking the higher-ranking answers in the thread, I think a key distinction hasn't been made: "introductory" for whom? A first course in statistics (that happens to have a Bayesian approach)? An introduction to Bayesian methods for someone with basic undergraduate (non-Bayesian) statistics classes? Or an introduction to Bayesian statistics for a practitioner of non-Bayesian statistics who has finally been persuaded that this Bayesian thing isn't a fad? Very different introductions.
John Kruschke released a book in mid 2011 called Doing Bayesian Data Analysis: A Tutorial with R and BUGS. (A second edition was released in Nov 2014: Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan.) It is truly introductory. If you want to walk from frequentist stats into Bayes though, especially with multilevel modelling, I recommend Gelman and Hill.
@Amir's suggestion is a duplicate of this. (The full title of the book is "Doing Bayesian Data Analysis: A Tutorial with R and BUGS".) As a truly introductory book, I've +1'd each.
I also vote for Kruschke's book. I've browsed most of the books listed in the answers and this is the one I found most clear. IMO, it is the most clear stats book I have read. It helps a lot that R code is available to match formulas with code. The author starts with very simple examples and builds on them. Very little background is needed. All reviews on Amazon are highly favorable. Hoff's book is my second favorite.
Haha, I like the book cover: "Why the happy puppies? (as if happy puppies needed justification!)"
My vote also goes to Kruschke's 2010 book. In trying to learn Bayesian statistics, I tried several of them, and this one hit the mark. Hard.
I read the first 8 chapters of this book based on this recommendation, and then I just hit a wall. This book has an odd mixture of writing like it's an introductory book, but then making sudden unexplained jumps in mathematics with no explanation that just leave you scratching your head. Statistical Rethinking, which is mentioned further down this page, does a *much* better job of helping to develop an intuition. You almost need to read it before trying to get through later chapters of Kruschke.
My favorite is "Bayesian Data Analysis" by Gelman, et al. (The pdf version is legally free since April 2020!)
Gelman et al. is well-regarded but explictly intended for a graduate course. If you don't have substantial prior coursework in statistics, it is largely a waste.
This is an introductory book for people who have a decent amount of statistical background already.
I started a PhD in Statistics 9 months ago and to be honest Gelman's BDA is still above me, so I wouldn't call it an introductory text!
-1, because according to multiple comments and other answers, this isn't introductory.
The first four or five chapters are truly introductory! so belongs here.
@stucash that's an emotive approach to discussing a disagreement. I guess it depends what kind of introduction you need. When someone asks me for an introduction, the response is to point them to the most compact version of the math needed to actually do something in the area. If that looks like too much work, you then need to question what exactly you get from any approach which requires less work.
@conjectures Thanks for taking time to reply however I personally found your comment emotive as well. I do understand everyone has their own definition of being introductory but without required details in the answer and the original question, it is hard to say which version of "being introductory" should be applied in this context. That being said, I found naught101's version is as right as yours.
Statistical Rethinking, has been released just a few weeks ago and hence I am still reading it, but I think is a very nice and fresh addition to the really introductory books about Bayesian Statistics. The author uses a similar approach as the one used by John Kruschke in his puppy books; very verbose, detailed explanations, nice pedagogical examples, he also uses a computational rather than mathematical approach.
Youtube lectures and other material is also available from here.
+1 I'm listening through the lectures now. He's very entertaining, and has a good approach. The book is excellent and takes you from basics to hierarchical models. It only assumes that the reader is somewhat scientific, has a reasonable grasp of mathematics (not including calculus) and has heard some things about statistics. It's the book I wish I'd had. The order he presents things in, and his system of asides is brilliant.
I hit a wall trying to work through Kruschke's book where he starting making some big leaps in logic that I just couldn't follow. Luckily, I came across Statistical Rethinking, which so far is the only book I've found that gives you a genuinely intuitive understanding of the topic.
After going through the thread, I tried reading the first chapter of this book, and I found it very difficult as a _non-native English speaker_ and as a _non-scientist_. First I had to go through the words like _epistemology_, _idiosyncratic_, then there are long sentences, which I had to read twice/thrice to understand what tehy means literally (forget about the conclusion of those sentences). Then the very first example is about natural evolution, which sounded Greek to me: _number of sites, number of alleles, neutrality_. The book could be easy for a lot, but could be difficult for many
Sivia and Skilling, Data analysis: a Bayesian tutorial (2ed) 2006 246p 0198568320 books.goo:
Statistics lectures have been a source of much bewilderment and frustration for generations of students. This book attempts to remedy the situation by expounding a logical and unified approach to the whole subject of data analysis. This text is intended as a tutorial guide for senior undergraduates and research students in science and engineering ...
I don't know the other recommendations though.
For an introduction, I would recommend Probabilistic Programming & Bayesian Methods for Hackers by Cam Davidson-Pilon, freely available online.
From its description:
An intro to Bayesian methods and probabilistic programming from a computation/understanding-first, mathematics-second point of view.
It's highly visual, cuts straight to the value and backfills gritty details later, has lots of examples, has interactive code (in IPython Notebook).
I thoroughly recommend the entertaining polemic "Probability Theory: The Logic of Science" by E.T. Jaynes.
This is an introductory text in the sense of not requiring (and in fact preferring) no previous knowledge of statistics, but it does eventually employ fairly sophisticated mathematics. Compared to most of the other answers provided, this book is not nearly as practical or easy to digest, rather it provides the philosophical bedrock to why you would want to employ Bayesian methods, and why not to use frequentist approaches. It is introductory in a historical and philosophical, but not pedagogical way.
This is a brilliant book about Bayesian *thinking* rather than applying Bayesian methods. I think this is a good companion text to something which goes more into how do Bayesian computations.
That's a good way of putting it. I think Sivia and Skilling is an ideal companion text for introducing the methods in practice (which has already been suggested in another answer).
I am an electrical engineer and not a statistician. I spent a lot of time to go through Gelman but I don't think one can refer to Gelman as introductory at all. My bayesian-guru professor from Carnegie Mellon agrees with me on this. having the minimum knowledge of statistics and R and Bugs(as the easy way to DO something with Bayesian stat) Doing Bayesian Data Analysis: A Tutorial with R and BUGS is an amazing start. You can compare all offered books easily by their book cover!
5 years later update: I want to add that perhaps one other major way of learning in a fast way(40 mins) is to go through the documentation of a Bayesian Net GUI based tool such as Netica2. It starts with basics, walks you through the steps of building a net based on a situation and data, and how to run your own questions back and forth to "get it!".
The Gelman books are all excellent but not necessarily introductory in that they assume that you know some statistics already. Therefore they are an introduction to the Bayesian way of doing statistics rather than to statistics in general. I would still give them the thumbs up, however.
As an introductory statistics/econometrics book which takes a Bayesian perspective, I would recommend Gary Koop's Bayesian Econometrics.
Its focus isn't strictly on Bayesian statistics, so it lacks some methodology, but David MacKay's Information Theory, Inference, and Learning Algorithms made me intuitively grasp Bayesian statistics better than others - most do the how quite nicely, but I felt MacKay explained why better.
And it is available for free download at the authors page: http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
Like Sivia, this is very nice if you have some physics background and can be rough if not. Not a good guide to any kind of applied social statistics (for that use Gelman and Hill, or Gelman et al. above) but really *great* for prompting you to really think about the core issues.