The Search for Certainty: On the Clash of Science and Philosophy of Probability (269 Pages)

This volume represents a radical departure from the current philosophical duopoly in the area of foundations of probability, that is, the frequency and subjective theories. One of the main new ideas is a set of scientific laws of probability. The new laws are simple, intuitive and, last but not least, they agree well with the contents of current textbooks on probability. Another major new claim is that the "frequency statistics" has nothing in common with the "frequency philosophy of probability," contrary to popular belief. Similarly, contrary to the general perception, the "Bayesian statistics" shares nothing in common with the "subjective philosophy of probability." The book is non-partisan on the scientific side -- it is supportive of both frequency statistics and Bayesian statistics. On the other hand, it contains well-documented and thoroughly-explained criticisms of the frequency and subjective philosophies of probability. Short reviews of other philosophical theories of probability and basic mathematical methods of probability and statistics are incorporated. The book includes substantial chapters on decision theory and teaching probability, and it is easily accessible to the general audience. Contents: Main Philosophies of Probability The Science of Probability Decision Making The Frequency Philosophy of Probability Classical Statistics The Subjective Philosophy of Probability Bayesian Statistics Teaching Probability Abuse of Language What is Science? What is Philosophy? Concluding Remarks Mathematical Methods of Probability and Statistics Literature Review Contents......Page 10 Preface......Page 8 1.1 Reality and Philosophy......Page 16 1.2 Summary of the Main Claims......Page 17 1.2.1 Critique of the frequency and subjective theories......Page 18 1.2.2 Scienti c laws of probability......Page 21 1.2.3 Statistics and philosophy......Page 23 1.3 Historical and Social Context......Page 25 1.4 Disclaimers......Page 27 2. Main Philosophies of Probability......Page 30 2.2 The Logical Theory......Page 31 2.4 The Subjective Theory......Page 33 2.4.1 Interpreting subjectivity......Page 34 2.4.2 Verification of probabilistic statements......Page 35 2.4.3 Subjectivity as an escape from the shackles of verification......Page 37 2.4.4 The Dutch book argument......Page 38 2.4.5 The axiomatic system......Page 40 2.4.6 Identification of probabilities and decisions......Page 41 2.5 The Frequency Theory......Page 42 2.6 Summary of Philosophical Theories of Probability......Page 44 2.7 Incompleteness—The Universal Malady......Page 45 2.8 Popular Philosophy......Page 46 3. The Science of Probability......Page 50 3.1 Interpretation of (L1)-(L5)......Page 51 3.2 A Philosophy of Probability and Scienti c Veri cation of (L1)-(L5)......Page 55 3.3 Predictions......Page 58 3.4 Is Symmetry Objective?......Page 68 3.5 Symmetry is Relative......Page 69 3.6 Moderation is Golden......Page 70 3.6.1 A sixth law?......Page 72 3.7 Circularity in Science and Philosophy......Page 73 3.8 Applications of (L1)-(L5): Some Examples......Page 74 3.8.2 Laws (L1)-(L5) as a basis for statistics......Page 75 3.8.3 Long run frequencies and (L1)-(L5)......Page 76 3.8.4 Life on Mars......Page 77 3.9 Symmetry and Data......Page 79 3.10 Probability of a Single Event......Page 80 3.11 On Events that Belong to Two Sequences......Page 81 3.12 Deformed Coins......Page 82 3.13 Symmetry and Theories of Probability......Page 83 3.14 Are Coin Tosses i.i.d. or Exchangeable?......Page 85 3.15 Physical and Epistemic Probabilities......Page 86 3.16 Countable Additivity......Page 87 3.17 Quantum Mechanics......Page 89 4.1 Decision Making in the Context of (L1)-(L5)......Page 92 4.1.1 Maximization of expected gain......Page 93 4.1.2 Maximization of expected gain as an axiom......Page 95 4.1.3 Stochastic ordering of decisions......Page 96 4.1.4 Generating predictions......Page 98 4.1.5 A new prisoner paradox......Page 99 4.2 Events with No Probabilities......Page 101 4.3 Law Enforcement......Page 103 4.4 Utility in Complex Decision Problems......Page 105 4.4.2 Nonlinearity of utility......Page 106 4.4.3 Utility of non-monetary rewards......Page 108 4.4.4 Unobservable utilities......Page 109 4.5 Identification of Decisions and Probabilities......Page 110 5. The Frequency Philosophy of Probability......Page 112 5.2 Inconsistencies in von Mises' Theory......Page 113 5.3 Collective as an Elementary Concept......Page 115 5.4 Applications of Probability Do Not Rely on Collectives......Page 116 5.5 Collectives in Real Life......Page 118 5.6 Collectives and Symmetry......Page 120 5.7 Frequency Theory and the Law of Large Numbers......Page 121 5.8 Benefits of Imagination and Imaginary Bene ts......Page 122 5.9 Imaginary Collectives......Page 123 5.10 Computer Simulations......Page 124 5.11 Frequency Theory and Individual Events......Page 125 5.12 Collectives and Populations......Page 126 5.13 Are All i.i.d. Sequences Collectives?......Page 127 5.14 Are Collectives i.i.d. Sequences?......Page 128 6.1 Confidence Intervals......Page 130 6.2 Estimation......Page 133 6.3 Hypothesis Testing......Page 136 6.3.1 Hypothesis tests and collectives......Page 137 6.3.2 Hypothesis tests and the frequency interpretation of probability......Page 138 6.3.3 Hypothesis testing and (L1)-(L5)......Page 139 6.4 Experimental Statistics—A Missing Science......Page 141 6.6 Does Classical Statistics Need the Frequency Theory?......Page 144 7.1 The Smoking Gun......Page 146 7.2 How to Eat the Cake and Have It Too......Page 148 7.3 The Subjective Theory of Probability is Objective......Page 151 7.4 A Science without Empirical Content......Page 152 7.5.1 Creating something out of nothing......Page 154 7.5.2 The essence of probability......Page 155 7.6 The Subjective Theory Does Not Imply the Bayes Theorem......Page 158 7.6.1 Sequential decisions in statistics......Page 159 7.6.2 Honest mistakes......Page 160 7.6.3 The past and the future are decoupled......Page 162 7.6.4 The Dutch book argument is static......Page 165 7.6.5 Cohabitation with an evil demiurge......Page 168 7.6.6 The Bayes theorem is unobservable......Page 170 7.6.7 All statistical strategies are Bayesian......Page 171 7.7 The Dutch Book Argument is Rejected by Bayesians......Page 173 7.8 No Need to Collect Data......Page 174 7.9 Empty Promises......Page 175 7.10 The Meaning of Consistency......Page 176 7.11 Interpreting Miracles......Page 177 7.12 Science, Probability and Subjectivism......Page 178 7.13 A Word with a Thousand Meanings......Page 180 7.14 Apples and Oranges......Page 184 7.15 Arbitrage......Page 186 7.16 Subjective Theory and Atheism......Page 187 7.17 Imagination and Probability......Page 188 7.18 A Misleading Slogan......Page 190 7.19 Axiomatic System as a Magical Trick......Page 191 8.1.1 Non-existence vs. informal assessment......Page 192 8.1.3 Conditioning vs. individuality......Page 193 8.2 Elements of Bayesian Analysis......Page 194 8.3 Models......Page 195 8.4 Priors......Page 196 8.4.1 Objective priors......Page 197 8.4.2 Bayesian statistics as an iterative method......Page 198 8.4.3 Truly subjective priors......Page 199 8.6 Posteriors......Page 202 8.6.1 Non-convergence of posterior distributions......Page 203 8.8 Spurious Predictions......Page 205 8.9 Who Needs Subjectivism?......Page 206 8.10 Preaching to the Converted......Page 207 8.11 Constants and Random Variables......Page 210 8.12 Criminal Trials......Page 211 9. Teaching Probability......Page 214 9.1 Teaching Independence......Page 217 9.2 Probability and Frequency......Page 218 9.3 Undergraduate Textbooks......Page 219 10. Abuse of Language......Page 222 11. What is Science?......Page 226 11.1 From Intuition to Science......Page 229 11.2 Science as Service......Page 231 11.3 Decision Making......Page 232 11.4 Mathematical Foundations of Probability......Page 233 11.5 Axioms versus Laws of Science......Page 235 12. What is Philosophy?......Page 236 12.1 What is Philosophy of Probability?......Page 238 12.2 Is Probability a Science?......Page 241 12.3 Objective and Subjective Probabilities......Page 242 12.4 Yin and Yang......Page 243 12.5 What Exists?......Page 244 12.6 Who Needs Philosophy?......Page 245 13.1 Does Science Have to be Rational?......Page 246 13.2 Common Elements in Frequency and Subjective Theories......Page 247 13.4 Common Misconceptions......Page 248 14.1 Probability......Page 252 14.1.1 Law of Large Numbers, Central Limit Theorem and Large Deviations Principle......Page 253 14.1.2 Exchangeability and de Finetti's theorem......Page 254 14.2 Classical Statistics......Page 255 14.3 Bayesian Statistics......Page 256 14.4 Contradictory Predictions......Page 257 15.1 Classics......Page 260 15.3 Philosophy and Mathematics......Page 261 Bibliography......Page 264 Index......Page 268 Contents 10 Preface 8 1. Introduction 16 1.1 Reality and Philosophy 16 1.2 Summary of the Main Claims 17 1.2.1 Critique of the frequency and subjective theories 18 1.2.2 Scienti c laws of probability 21 1.2.3 Statistics and philosophy 23 1.3 Historical and Social Context 25 1.4 Disclaimers 27 2. Main Philosophies of Probability 30 2.1 The Classical Theory 31 2.2 The Logical Theory 31 2.3 The Propensity Theory 33 2.4 The Subjective Theory 33 2.4.1 Interpreting subjectivity 34 2.4.2 Verification of probabilistic statements 35 2.4.3 Subjectivity as an escape from the shackles of verification 37 2.4.4 The Dutch book argument 38 2.4.5 The axiomatic system 40 2.4.6 Identification of probabilities and decisions 41 2.4.7 The Bayes theorem 42 2.5 The Frequency Theory 42 2.6 Summary of Philosophical Theories of Probability 44 2.7 Incompleteness—The Universal Malady 45 2.8 Popular Philosophy 46 3. The Science of Probability 50 3.1 Interpretation of (L1)-(L5) 51 3.2 A Philosophy of Probability and Scienti c Veri cation of (L1)-(L5) 55 3.3 Predictions 58 3.4 Is Symmetry Objective? 68 3.5 Symmetry is Relative 69 3.6 Moderation is Golden 70 3.6.1 A sixth law? 72 3.7 Circularity in Science and Philosophy 73 3.8 Applications of (L1)-(L5): Some Examples 74 3.8.1 Poisson process 75 3.8.2 Laws (L1)-(L5) as a basis for statistics 75 3.8.3 Long run frequencies and (L1)-(L5) 76 3.8.4 Life on Mars 77 3.9 Symmetry and Data 79 3.10 Probability of a Single Event 80 3.11 On Events that Belong to Two Sequences 81 3.12 Deformed Coins 82 3.13 Symmetry and Theories of Probability 83 3.14 Are Coin Tosses i.i.d. or Exchangeable? 85 3.15 Physical and Epistemic Probabilities 86 3.16 Countable Additivity 87 3.17 Quantum Mechanics 89 4. Decision Making 92 4.1 Decision Making in the Context of (L1)-(L5) 92 4.1.1 Maximization of expected gain 93 4.1.2 Maximization of expected gain as an axiom 95 4.1.3 Stochastic ordering of decisions 96 4.1.4 Generating predictions 98 4.1.5 A new prisoner paradox 99 4.2 Events with No Probabilities 101 4.3 Law Enforcement 103 4.4 Utility in Complex Decision Problems 105 4.4.1 Variability of utility in time 106 4.4.2 Nonlinearity of utility 106 4.4.3 Utility of non-monetary rewards 108 4.4.4 Unobservable utilities 109 4.5 Identification of Decisions and Probabilities 110 5. The Frequency Philosophy of Probability 112 5.1 The Smoking Gun 113 5.2 Inconsistencies in von Mises' Theory 113 5.3 Collective as an Elementary Concept 115 5.4 Applications of Probability Do Not Rely on Collectives 116 5.5 Collectives in Real Life 118 5.6 Collectives and Symmetry 120 5.7 Frequency Theory and the Law of Large Numbers 121 5.8 Benefits of Imagination and Imaginary Bene ts 122 5.9 Imaginary Collectives 123 5.10 Computer Simulations 124 5.11 Frequency Theory and Individual Events 125 5.12 Collectives and Populations 126 5.13 Are All i.i.d. Sequences Collectives? 127 5.14 Are Collectives i.i.d. Sequences? 128 6. Classical Statistics 130 6.1 Confidence Intervals 130 6.2 Estimation 133 6.2.1 Estimation and (L1)-(L5) 136 6.3 Hypothesis Testing 136 6.3.1 Hypothesis tests and collectives 137 6.3.2 Hypothesis tests and the frequency interpretation of probability 138 6.3.3 Hypothesis testing and (L1)-(L5) 139 6.4 Experimental Statistics—A Missing Science 141 6.5 Hypothesis Testing and (L5) 144 6.6 Does Classical Statistics Need the Frequency Theory? 144 7. The Subjective Philosophy of Probability 146 7.1 The Smoking Gun 146 7.2 How to Eat the Cake and Have It Too 148 7.3 The Subjective Theory of Probability is Objective 151 7.4 A Science without Empirical Content 152 7.5 The Weakest Scientific Theory Ever 154 7.5.1 Creating something out of nothing 154 7.5.2 The essence of probability 155 7.6 The Subjective Theory Does Not Imply the Bayes Theorem 158 7.6.1 Sequential decisions in statistics 159 7.6.2 Honest mistakes 160 7.6.3 The past and the future are decoupled 162 7.6.4 The Dutch book argument is static 165 7.6.5 Cohabitation with an evil demiurge 168 7.6.6 The Bayes theorem is unobservable 170 7.6.7 All statistical strategies are Bayesian 171 7.7 The Dutch Book Argument is Rejected by Bayesians 173 7.8 No Need to Collect Data 174 7.9 Empty Promises 175 7.10 The Meaning of Consistency 176 7.11 Interpreting Miracles 177 7.12 Science, Probability and Subjectivism 178 7.13 A Word with a Thousand Meanings 180 7.14 Apples and Oranges 184 7.15 Arbitrage 186 7.16 Subjective Theory and Atheism 187 7.17 Imagination and Probability 188 7.18 A Misleading Slogan 190 7.19 Axiomatic System as a Magical Trick 191 8. Bayesian Statistics 192 8.1 Two Faces of Subjectivity 192 8.1.1 Non-existence vs. informal assessment 192 8.1.2 Are all probabilities subjective? 193 8.1.3 Conditioning vs. individuality 193 8.1.4 Nonexistent decisions 194 8.2 Elements of Bayesian Analysis 194 8.3 Models 195 8.4 Priors 196 8.4.1 Objective priors 197 8.4.2 Bayesian statistics as an iterative method 198 8.4.3 Truly subjective priors 199 8.5 Data 202 8.6 Posteriors 202 8.6.1 Non-convergence of posterior distributions 203 8.7 Bayesian Statistics and (L1)-(L5) 205 8.8 Spurious Predictions 205 8.9 Who Needs Subjectivism? 206 8.10 Preaching to the Converted 207 8.11 Constants and Random Variables 210 8.12 Criminal Trials 211 9. Teaching Probability 214 9.1 Teaching Independence 217 9.2 Probability and Frequency 218 9.3 Undergraduate Textbooks 219 10. Abuse of Language 222 11. What is Science? 226 11.1 From Intuition to Science 229 11.2 Science as Service 231 11.3 Decision Making 232 11.4 Mathematical Foundations of Probability 233 11.5 Axioms versus Laws of Science 235 12. What is Philosophy? 236 12.1 What is Philosophy of Probability? 238 12.2 Is Probability a Science? 241 12.3 Objective and Subjective Probabilities 242 12.4 Yin and Yang 243 12.5 What Exists? 244 12.6 Who Needs Philosophy? 245 13. Concluding Remarks 246 13.1 Does Science Have to be Rational? 246 13.2 Common Elements in Frequency and Subjective Theories 247 13.3 On Peaceful Coexistence 248 13.4 Common Misconceptions 248 14. Mathematical Methods of Probability and Statistics 252 14.1 Probability 252 14.1.1 Law of Large Numbers, Central Limit Theorem and Large Deviations Principle 253 14.1.2 Exchangeability and de Finetti's theorem 254 14.2 Classical Statistics 255 14.3 Bayesian Statistics 256 14.4 Contradictory Predictions 257 15. Literature Review 260 15.1 Classics 260 15.2 Philosophy 261 15.3 Philosophy and Mathematics 261 Bibliography 264 Index 268 1. Introduction. 1.1. Reality and philosophy. 1.2. Summary of the main claims. 1.3. Historical and social context. 1.4. Disclaimers -- 2. Main Philosophies of probability. 2.1. The classical theory. 2.2. The logical theory. 2.3. The propensity theory. 2.4. The subjective theory. 2.5. The frequency theory. 2.6. Summary of philosophical theories of probability. 2.7. Incompleteness - the universal malady. 2.8. Popular philosophy -- 3. The science of probability. 3.1. Interpretation of (L1)-(L5). 3.2. A philosophy of probability and scientific verification of (L1)-(L5). 3.3. Predictions. 3.4. Is symmetry objective? 3.5. Symmetry is relative. 3.6. Moderation is golden. 3.7. Circularity in science and philosophy. 3.8. Applications of (L1)-(L5) : some examples. 3.9. Symmetry and data. 3.10. Probability of a single event. 3.11. On events that belong to two sequences. 3.12. Deformed coins. 3.13. Symmetry and theories of probability. 3.14. Are coin tosses i.i.d. or exchangeable? 3.15. Physical and epistemic probabilities. 3.16. Countable additivity. 3.17. Quantum mechanics -- 4. Decision making. 4.1. Decision making in the context of (L1)-(L5). 4.2. Events with no probabilities. 4.3. Law enforcement. 4.4. Utility in complex decision problems. 4.5. Identification of decisions and probabilities -- 5. The frequency philosophy of probability. 5.1. The smoking gun. 5.2. Inconsistencies in von Mises' theory. 5.3. Collective as an elementary concept. 5.4. Applications of probability do not rely on collectives. 5.5. Collectives in real life. 5.6. Collectives and symmetry. 5.7. Frequency theory and the law of large numbers. 5.8. Benefits of imagination and imaginary benefits. 5.9. Imaginary collectives. 5.10. Computer simulations. 5.11. Frequency theory and individual events. 5.12. Collectives and populations. 5.13. Are all i.i.d. sequences collectives? 5.14. Are collectives i.i.d. sequences? -- 6. Classical statistics. 6.1. Confidence intervals. 6.2. Estimation. 6.3. Hypothesis testing. 6.4. Experimental statistics - a missing science. 6.5. Hypothesis testing and (L5) 6.6. Does classical statistics need the frequency theory? -- 7. The subjective philosophy of probability. 7.1. The smoking gun. 7.2. How to eat the cake and have it too. 7.3. The subjective theory of probability is objective. 7.4. A science without empirical content. 7.5. The weakest scientific theory ever. 7.6. The subjective theory does not imply the Bayes theorem. 7.7. The Dutch book argument is rejected by Bayesians. 7.8. No need to collect data. 7.9. Empty promises. 7.10. The meaning of consistency. 7.11. Interpreting miracles. 7.12. Science, probability and subjectivism. 7.13. A word with a thousand meanings. 7.14. Apples and oranges. 7.15. Arbitrage. 7.16. Subjective theory and atheism. 7.17. Imagination and probability. 7.18. A misleading slogan. 7.19. Axiomatic system as a magical trick -- 8. Bayesian statistics. 8.1. Two faces of subjectivity. 8.2. Elements of Bayesian analysis. 8.3. Models. 8.4. Priors. 8.5. Data. 8.6. Posteriors. 8.7. Bayesian statistics and (L1)-(L5). 8.8. Spurious predictions. 8.9. Who needs subjectivism? 8.10. Preaching to the converted. 8.11. Constants and random variables. 8.12. Criminal trials -- 9. Teaching probability. 9.1. Teaching independence. 9.2. Probability and frequency. 9.3. Undergraduate textbooks -- 10. Abuse of language -- 11. What is science? 11.1. From intuition to science. 11.2. Science as service. 11.3. Decision making. 11.4. Mathematical foundations of probability. 11.5. Axioms versus laws of science -- 12. What is philosophy? 12.1. What is philosophy of probability? 12.2. Is probability a science? 12.3. Objective and subjective probabilities. 12.4. Yin and yang. 12.5. What exists? 12.6. Who needs philosophy? -- 13. Concluding remarks. 13.1. Does science have to be rational? 13.2. Common elements in frequency and subjective theories. 13.3. On peaceful coexistence. 13.4. Common misconceptions -- 14. Mathematical methods of probability and statistics. 14.1. Probability. 14.2. Classical statistics. 14.3. Bayesian statistics. 14.4. Contradictory predictions -- 15. Literature review. 15.1. Classics. 15.2. Philosophy. 15.3. Philosophy and mathematics This volume represents a radical departure from the current philosophical duopoly in the area of foundations of probability, that is, the frequency and subjective theories. One of the main new ideas is a set of scientific laws of probability. The new laws are simple, intuitive and, last but not least, they agree well with the contents of current textbooks on probability. Another major new claim is that the frequency statistics has nothing in common with the frequency philosophy of probability, contrary to popular belief. Similarly, contrary to the general perception, the Bayesian statistics shares nothing in common with the subjective philosophy of probability. This volume is non-partisan on the scientific side that it is supportive of both frequency statistics and Bayesian statistics. On the other hand, it contains well-documented and thoroughly-explained criticisms of the frequency and subjective philosophies of probability. Short reviews of other philosophical theories of probability and basic mathematical methods of probability and statistics are incorporated. This volume includes essential chapters on decision theory and teaching probability, and it is easily accessible to the general audience.