Doing Bayesian data analysis : a tutorial introduction with R and BUGS

There is an explosion of interest in Bayesian statistics, primarily because recently created computational methods have finally made Bayesian analysis tractable and accessible to a wide audience. Doing Bayesian Data Analysis, A Tutorial Introduction with R and BUGS, is for first year graduate students or advanced undergraduates and provides an accessible approach, as all mathematics is explained intuitively and with concrete examples. It assumes only algebra and 'rusty' calculus. Unlike other textbooks, this book begins with the basics, including essential concepts of probability and random sampling. The book gradually climbs all the way to advanced hierarchical modeling methods for realistic data. The text provides complete examples with the R programming language and BUGS software (both freeware), and begins with basic programming examples, working up gradually to complete programs for complex analyses and presentation graphics. These templates can be easily adapted for a large variety of students and their own research needs.The textbook bridges the students from their undergraduate training into modern Bayesian methods. -Accessible, including the basics of essential concepts of probability and random sampling -Examples with R programming language and BUGS software -Comprehensive coverage of all scenarios addressed by non-bayesian textbooks- t-tests, analysis of variance (ANOVA) and comparisons in ANOVA, multiple regression, and chi-square (contingency table analysis). -Coverage of experiment planning -R and BUGS computer programming code on website -Exercises have explicit purposes and guidelines for accomplishment Foreword......Page 1 Contents......Page 3 1.1 Real people can read this book......Page 13 1.2 Prerequisites......Page 14 1.3 The organization of this book......Page 15 1.3.2 Where’s the equivalent of traditional test X in this book?......Page 16 1.5 Acknowledgments......Page 17 I The Basics: Parameters, Probability, Bayes’ Rule, and R......Page 19 2 Introduction: Models we believe in......Page 21 2.1 Models of observations and models of beliefs......Page 22 2.1.1 Models have parameters......Page 23 2.2.1 Estimation of parameter values......Page 25 2.2.3 Model comparison......Page 26 2.3.2 Invoking R and using the command line......Page 27 2.3.3 A simple example of R in action......Page 28 2.3.4 Getting help in R......Page 29 2.3.5.2 Variable names in R......Page 30 2.4 Exercises......Page 31 3 What is this stuff called probability?......Page 33 3.1.1 Coin flips: Why you should care......Page 34 3.2.1.1 Simulating a long-run relative frequency......Page 35 3.2.1.2 Deriving a long-run relative frequency......Page 36 3.2.2.1 Calibrating a subjective belief by preferences......Page 37 3.3 Probability distributions......Page 38 3.3.2 Continuous distributions: Rendezvous with density......Page 39 3.3.2.1 Properties of probability density functions......Page 41 3.3.2.2 The normal probability density function......Page 42 3.3.3 Mean and variance of a distribution......Page 44 3.3.3.1 Mean as minimized variance......Page 45 3.3.5 Highest density interval (HDI)......Page 46 3.4 Two-way distributions......Page 47 3.4.1 Marginal probability......Page 48 3.4.2 Conditional probability......Page 50 3.4.3 Independence of attributes......Page 51 3.5.1 R code for Figure 3.1......Page 52 3.6 Exercises......Page 53 4 Bayes’ Rule......Page 4 4.1.1 Derived from definitions of conditional probability......Page 56 4.1.2 Intuited from a two-way discrete table......Page 57 4.2 Applied to models and data......Page 59 4.2.1 Data order invariance......Page 61 4.2.2 An example with coin flipping......Page 62 4.3.2 Prediction of data values......Page 64 4.3.3 Model comparison......Page 65 4.3.5.1 Holmesian deduction......Page 68 4.4.1 R code for Figure 4.1......Page 69 4.5 Exercises......Page 71 II All the Fundamentals Applied to Inferring a Binomial Proportion......Page 75 5 Inferring a Binomial Proportion via Exact Mathematical Analysis......Page 77 5.1 The likelihood function: Bernoulli distribution......Page 78 5.2 A description of beliefs: The beta distribution......Page 79 5.2.1 Specifying a beta prior......Page 80 5.2.2 The posterior beta......Page 82 5.3.1 Estimating the binomial proportion......Page 83 5.3.2 Predicting data......Page 84 5.3.3 Model comparison......Page 85 5.4 Summary: How to do Bayesian inference......Page 87 5.5.1 R code for Figure 5.2......Page 88 5.6 Exercises......Page 91 6 Inferring a Binomial Proportion via Grid Approximation......Page 95 6.2 Discretizing a continuous prior density......Page 96 6.2.1 Examples using discretized priors......Page 97 6.3 Estimation......Page 99 6.4 Prediction of subsequent data......Page 100 6.6 Summary......Page 101 6.7.1 R code for Figure 6.2 etc......Page 102 6.8 Exercises......Page 104 7 Inferring a Binomial Proportion via the Metropolis Algorithm......Page 109 7.1 A simple case of the Metropolis algorithm......Page 110 7.1.1 A politician stumbles upon the Metropolis algorithm......Page 111 7.1.3 General properties of a random walk......Page 113 7.1.5 Why it works......Page 116 7.2 The Metropolis algorithm more generally......Page 120 7.2.2 Terminology: Markov chain Monte Carlo......Page 121 7.3 From the sampled posterior to the three goals......Page 122 7.3.1.1 Highest density intervals from random samples......Page 123 7.3.1.2 Using a sample to estimate an integral......Page 124 7.3.3 Model comparison: Estimation of p(D)......Page 125 7.4 MCMC in BUGS......Page 127 7.4.1 Parameter estimation with BUGS......Page 128 7.4.2 BUGS for prediction......Page 130 7.4.3 BUGS for model comparison......Page 131 7.5 Conclusion......Page 132 7.6.1 R code for a home-grown Metropolis......Page 133 7.7 Exercises......Page 135 8 Inferring Two Binomial Proportions via Gibbs Sampling......Page 139 8.1 Prior, likelihood and posterior for two proportions......Page 141 8.2 The posterior via exact formal analysis......Page 142 8.3 The posterior via grid approximation......Page 145 8.4 The posterior via Markov chain Monte Carlo......Page 146 8.4.1 Metropolis algorithm......Page 147 8.4.2 Gibbs sampling......Page 148 8.4.2.1 Disadvantages of Gibbs sampling......Page 151 8.5 Doing it with BUGS......Page 152 8.5.1 Sampling the prior in BUGS......Page 153 8.6 How different are the underlying biases?......Page 154 8.7 Summary......Page 155 8.8.1 R code for grid approximation (Figures 8.1 and 8.2)......Page 156 8.8.2 R code for Metropolis sampler (Figure 8.3)......Page 158 8.8.3 R code for BUGS sampler (Figure 8.6)......Page 161 8.8.4 R code for plotting a posterior histogram......Page 163 8.9 Exercises......Page 165 9 Bernoulli Likelihood with Hierarchical Prior......Page 169 9.1 A single coin from a single mint......Page 170 9.1.1 Posterior via grid approximation......Page 173 9.2 Multiple coins from a single mint......Page 176 9.2.1 Posterior via grid approximation......Page 178 9.2.2 Posterior via Monte Carlo sampling......Page 181 9.2.2.1 Doing it with BUGS......Page 183 9.2.3 Outliers and shrinkage of individual estimates......Page 187 9.2.4 Case study: Therapeutic touch......Page 189 9.3.1 Independent mints......Page 190 9.3.2 Dependent mints......Page 194 9.3.3 Individual differences and meta-analysis......Page 196 9.5.1 Code for analysis of therapeutic-touch experiment......Page 197 9.5.2 Code for analysis of filtration-condensation experiment......Page 200 9.6 Exercises......Page 203 10.1 Model comparison as hierarchical modeling......Page 207 10.2.1 A simple example......Page 209 10.2.2 A realistic example with “pseudopriors”......Page 211 10.2.3 Some practical advice when using transdimensional MCMC withpseudopriors......Page 216 10.3 Model comparison and nested models......Page 218 10.4.1 Comparing methods for MCMC model comparison......Page 220 10.4.2 Summary and caveats......Page 221 10.5 Exercises......Page 222 11 Null Hypothesis Significance Testing......Page 227 11.1.1 When the experimenter intends to fix N......Page 229 11.1.2 When the experimenter intends to fix z......Page 231 11.1.3 Soul searching......Page 232 11.2 Prior knowledge about the coin......Page 234 11.2.2.1 Priors are overt and should influence......Page 235 11.3.1 NHST confidence interval......Page 236 11.4 Multiple comparisons......Page 239 11.4.1 NHST correction for experimentwise error......Page 240 11.4.2 Just one Bayesian posterior no matter how you look at......Page 242 11.5.1 Planning an experiment......Page 243 11.5.2 Exploring model predictions (posterior predictive check)......Page 244 11.6 Exercises......Page 245 12 Bayesian Approaches to Testing a Point (“Null”) Hypothesis......Page 251 12.1.1 Is a null value of a parameter among the credible values?......Page 252 12.1.2 Is a null value of a difference among the credible values?......Page 253 12.1.2.1 Differences of correlated parameters......Page 254 12.1.3 Region of Practical Equivalence (ROPE)......Page 256 12.2 The model-comparison (two-prior) approach......Page 257 12.2.1 Are the biases of two coins equal or not?......Page 258 12.2.1.1 Formal analytical solution......Page 259 12.2.1.2 Example application......Page 260 12.2.2 Are different groups equal or not?......Page 261 12.3.2 Recommendations......Page 263 12.4.1 R code for Figure 12.5......Page 264 12.5 Exercises......Page 267 13 Goals, Power, and Sample Size......Page 271 13.1.1 Goals and Obstacles......Page 272 13.1.2 Power......Page 273 13.1.3 Sample Size......Page 274 13.2 Sample size for a single coin......Page 276 13.2.1 When the goal is to exclude a null value......Page 277 13.2.2 When the goal is precision......Page 278 13.3 Sample size for multiple mints......Page 279 13.4 Power: prospective, retrospective, and replication......Page 281 13.4.1 Power analysis requires verisimilitude of simulated data......Page 282 13.5 The importance of planning......Page 283 13.6.1 Sample size for a single coin......Page 284 13.6.2 Power and sample size for multiple mints......Page 286 13.7 Exercises......Page 293 III The Generalized Linear Model......Page 301 14 Overview of the Generalized Linear Model......Page 303 14.1.1 Predictor and predicted variables......Page 304 14.1.2 Scale types: metric, ordinal, nominal......Page 305 14.1.3 Linear function of a single metric predictor......Page 306 14.1.4 Additive combination of metric predictors......Page 308 14.1.5 Nonadditive interaction of metric predictors......Page 310 14.1.6.1 Linear model for a single nominal predictor......Page 312 14.1.6.2 Additive combination of nominal predictors......Page 314 14.1.6.3 Nonadditive interaction of nominal predictors......Page 315 14.1.7 Linking combined predictors to the predicted......Page 316 14.1.7.1 The sigmoid (a.k.a. logistic) function......Page 317 14.1.7.2 The cumulative normal (a.k.a. Phi) function......Page 319 14.1.9 Formal expression of the GLM......Page 320 14.2 Cases of the GLM......Page 323 14.2.1 Two or more nominal variables predicting frequency......Page 325 14.3 Exercises......Page 327 15 Metric Predicted Variable on a Single Group......Page 329 15.1.1 Solution by mathematical analysis......Page 330 15.1.2 Approximation by MCMC in BUGS......Page 334 15.1.3 Outliers and robust estimation: The t distribution......Page 335 15.1.4 When the data are non-normal: Transformations......Page 338 15.2 Repeated measures and individual differences......Page 340 15.2.1 Hierarchical model......Page 342 15.2.2 Implementation in BUGS......Page 343 15.4.1 Estimating the mean and precision of a normal likelihood......Page 345 15.4.2 Repeated measures: Normal across and normal within......Page 347 15.5 Exercises......Page 350 16 Metric Predicted Variable with One Metric Predictor......Page 355 16.1 Simple linear regression......Page 356 16.1.1 The hierarchical model and BUGS code......Page 358 16.1.1.1 Standardizing the data for MCMC sampling......Page 359 16.1.1.2 Initializing the chains......Page 360 16.1.2 The posterior: How big is the slope?......Page 361 16.1.3 Posterior prediction......Page 362 16.2 Outliers and robust regression......Page 364 16.3 Simple linear regression with repeated measures......Page 366 16.4 Summary......Page 369 16.5.1 Data generator for height and weight......Page 370 16.5.2 BRugs: Robust linear regression......Page 371 16.5.3 BRugs: Simple linear regression with repeated measures......Page 374 16.6 Exercises......Page 378 17 Metric Predicted Variable with Multiple Metric Predictors......Page 383 17.1.1 The perils of correlated predictors......Page 384 17.1.2 The model and BUGS program......Page 387 17.1.3 The posterior: How big are the slopes?......Page 388 17.1.4 Posterior prediction......Page 390 17.2 Hyperpriors and shrinkage of regression coefficients......Page 391 17.2.1 Informative priors, sparse data, and correlated predictors......Page 393 17.3 Multiplicative interaction of metric predictors......Page 395 17.3.1 The hierarchical model and BUGS code......Page 396 17.3.2 Interpreting the posterior......Page 397 17.4 Which predictors should be included?......Page 400 17.5.1 Multiple linear regression......Page 402 17.5.2 Multiple linear regression with hyperprior on coefficients......Page 406 17.6 Exercises......Page 411 18 Metric Predicted Variable with One Nominal Predictor......Page 413 18.1 Bayesian oneway ANOVA......Page 414 18.1.1 The hierarchical prior......Page 415 18.1.2 Doing it with R and BUGS......Page 416 18.1.3 A worked example......Page 418 18.1.3.1 Contrasts and complex comparisons......Page 419 18.1.3.2 Is there a difference?......Page 420 18.2 Multiple comparisons......Page 421 18.3 Two group Bayesian ANOVA and the NHST t test......Page 424 18.4.1 Bayesian oneway ANOVA......Page 425 18.5 Exercises......Page 429 19 Metric Predicted Variable with Multiple Nominal Predictors......Page 433 19.1 Bayesian multi-factor ANOVA......Page 434 19.1.1 Interaction of nominal predictors......Page 435 19.1.2 The hierarchical prior......Page 436 19.1.3 An example in R and BUGS......Page 437 19.1.4.1 Metric predictors and ANCOVA......Page 440 19.1.4.2 Interaction contrasts......Page 441 19.1.5 Non-crossover interactions, rescaling, and homogeneous variances......Page 442 19.2 Repeated measures, a.k.a. within-subject designs......Page 444 19.2.1 Why use a within-subject design? And why not?......Page 446 19.3.1 Bayesian two-factor ANOVA......Page 447 19.4 Exercises......Page 456 20 Dichotomous Predicted Variable......Page 461 20.1 Logistic regression......Page 462 20.1.2 Doing it in R and BUGS......Page 463 20.1.3 Interpreting the posterior......Page 464 20.1.6 Hyperprior across regression coefficients......Page 466 20.2 Interaction of predictors in logistic regression......Page 467 20.3 Logistic ANOVA......Page 468 20.4 Summary......Page 470 20.5.1 Logistic regression code......Page 471 20.5.2 Logistic ANOVA code......Page 475 20.6 Exercises......Page 480 21 Ordinal Predicted Variable......Page 483 21.1.2 The mapping from metric x to ordinal y......Page 484 21.1.3 The parameters and their priors......Page 486 21.1.5 Posterior prediction......Page 487 21.2 Some examples......Page 488 21.2.1 Why are some thresholds outside the data?......Page 490 21.3 Interaction......Page 492 21.5 R code......Page 493 21.6 Exercises......Page 498 22 Contingency Table Analysis......Page 501 22.1.2 The exponential link function......Page 502 22.1.3 The Poisson likelihood......Page 505 22.1.4 The parameters and the hierarchical prior......Page 506 22.2.1 Credible intervals on cell probabilities......Page 507 22.3 Log linear models for contingency tables......Page 508 22.4 R code for Poisson exponential model......Page 509 22.5 Exercises......Page 516 23 Tools in the Trunk......Page 519 23.1.1 Essential points......Page 520 23.1.3 Helpful points......Page 521 23.2 MCMC burn-in and thinning......Page 522 23.3.2 R code for computing HDI of a MCMC sample......Page 525 23.3.3 R code for computing HDI of a function......Page 527 23.4.1 Examples......Page 528 23.4.2 Reparameterization of two parameters......Page 529 References......Page 531 Index......Page 539 Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan, Second Edition provides an accessible approach for conducting Bayesian data analysis, as material is explained clearly with concrete examples. Included are step-by-step instructions on how to carry out Bayesian data analyses in the popular and free software R and WinBugs, as well as new programs in JAGS and Stan. The new programs are designed to be much easier to use than the scripts in the first edition. In particular, there are now compact high-level scripts that make it easy to run the programs on your own data sets. The book is divided into three parts and begins with the basics: models, probability, Bayes’ rule, and the R programming language. The discussion then moves to the fundamentals applied to inferring a binomial probability, before concluding with chapters on the generalized linear model. Topics include metric-predicted variable on one or two groups; metric-predicted variable with one metric predictor; metric-predicted variable with multiple metric predictors; metric-predicted variable with one nominal predictor; and metric-predicted variable with multiple nominal predictors. The exercises found in the text have explicit purposes and guidelines for accomplishment. This book is intended for first-year graduate students or advanced undergraduates in statistics, data analysis, psychology, cognitive science, social sciences, clinical sciences, and consumer sciences in business. Accessible, including the basics of essential concepts of probability and random sampling Examples with R programming language and JAGS software Comprehensive coverage of all scenarios addressed by non-Bayesian textbooks: t-tests, analysis of variance (ANOVA) and comparisons in ANOVA, multiple regression, and chi-square (contingency table analysis) Coverage of experiment planning R and JAGS computer programming code on website Exercises have explicit purposes and guidelines for accomplishment Provides step-by-step instructions on how to conduct Bayesian data analyses in the popular and free software R and WinBugs

There is an explosion of interest in Bayesian statistics, primarily because recently created computational methods have finally made Bayesian analysis tractable and accessible to a wide audience. Doing Bayesian Data Analysis, A Tutorial Introduction with R and BUGS, is for first year graduate students or advanced undergraduates and provides an accessible approach, as all mathematics is explained intuitively and with concrete examples. It assumes only algebra and ‘rusty’ calculus. Unlike other textbooks, this book begins with the basics, including essential concepts of probability and random sampling. The book gradually climbs all the way to advanced hierarchical modeling methods for realistic data. The text provides complete examples with the R programming language and BUGS software (both freeware), and begins with basic programming examples, working up gradually to complete programs for complex analyses and presentation graphics. These templates can be easily adapted for a large variety of students and their own research needs.The textbook bridges the students from their undergraduate training into modern Bayesian methods.

-Accessible, including the basics of essential concepts of probability and random sampling

-Examples with R programming language and BUGS software

-Comprehensive coverage of all scenarios addressed by non-bayesian textbooks- t-tests, analysis of variance (ANOVA) and comparisons in ANOVA, multiple regression, and chi-square (contingency table analysis).

-Coverage of experiment planning

-R and BUGS computer programming code on website

-Exercises have explicit purposes and guidelines for accomplishment

"There is an explosion of interest in Bayesian statistics, primarily because recently created computational methods have finally made Bayesian analysis tractable and accessible to a wide audience. Doing Bayesian Data Analysis, A Tutorial Introduction with R and BUGS, is for first year graduate students or advanced undergraduates and provides an accessible approach, as all mathematics is explained intuitively and with concrete examples. It assumes only algebra and a rustya calculus. Unlike other textbooks, this book begins with the basics, including essential concepts of probability and random sampling. The book gradually climbs all the way to advanced hierarchical modeling methods for realistic data. The text provides complete examples with the R programming language and BUGS software (both freeware), and begins with basic programming examples, working up gradually to complete programs for complex analyses and presentation graphics. These templates can be easily adapted for a large variety of students and their own research needs.The textbook bridges the students from their undergraduate training into modern Bayesian methods." - Publisher's description. "There is an explosion of interest in Bayesian statistics, primarily because recently created computational methods have finally made Bayesian analysis tractable and accessible to a wide audience. Doing Bayesian Data Analysis, A Tutorial Introduction with R and BUGS, is for first year graduate students or advanced undergraduates and provides an accessible approach, as all mathematics is explained intuitively and with concrete examples. It assumes only algebra and a rustya calculus. Unlike other textbooks, this book begins with the basics, including essential concepts of probability and random sampling. The book gradually climbs all the way to advanced hierarchical modeling methods for realistic data. The text provides complete examples with the R programming language and BUGS software (both freeware), and begins with basic programming examples, working up gradually to complete programs for complex analyses and presentation graphics. These templates can be easily adapted for a large variety of students and their own research needs. The textbook bridges the students from their undergraduate training into modern Bayesian methods."--Publisher's description Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan, Second Edition provides an accessible approach for conducting Bayesian data analysis, as material is explained clearly with concrete examples. Included are step-by-step instructions on how to carry out Bayesian data analyses. Download Link : (http://readbux.com/download?i=0124058884) readbux.com/download?i=0124058884 0124058884 Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan PDF by John Kruschke