A Course in Mathematical Statistics

A Course in Mathematical Statistics, Second Edition , contains enough material for a year-long course in probability and statistics for advanced undergraduate or first-year graduate students, or it can be used independently for a one-semester (or even one-quarter) course in probability alone. It bridges the gap between high and intermediate level texts so students without a sophisticated mathematical background can assimilate a fairly broad spectrum of the theorems and results from mathematical statistics. The coverage is extensive, and consists of probability and distribution theory, and statistical inference. * Contains 25% new material * Includes the most complete coverage of sufficiency * Transformation of Random Vectors * Sufficiency / Completeness / Exponential Families * Order Statistics * Elements of Nonparametric Density Estimation * Analysis of Variance (ANOVA) * Regression Analysis * Linear Models A Course in Mathematical Statistics 4 Copyright Page 5 Contents 8 Preface to the Second Edition 16 Preface to the First Edition 19 Chapter 1. Basic Concepts of Set Theory 22 1.1 Some Definitions and Notation 22 1.2* Fields and σ-Fields 29 Chapter 2. Some Probabilistic Concepts and Results 35 2.1 Probability Functions and Some Basic Properties and Results 35 2.2 Conditional Probability 42 2.3 Independence 48 2.4 Combinatorial Results 55 2.5* Product Probability Spaces 66 2.6* The Probability of Matchings 68 Chapter 3. On Random Variables and Their Distributions 74 3.1 Some General Concepts 74 3.2 Discrete Random Variables (and Random Vectors) 76 3.3 Continuous Random Variables (and Random Vectors) 86 3.4 The Poisson Distribution as an Approximation to the Binomial Distribution and the Binomial Distribution as an Approximation to the Hypergeometric Distribution 100 3.5* Random Variables as Measurable Functions and Related Results 103 Chapter 4. Distribution Functions, Probability Densities, and Their Relationship 106 4.1 The Cumulative Distribution Function (c.d.f. or d.f.) of a Random Vector—Basic Properties of the d.f. of a Random Variable 106 4.2 The d.f. of a Random Vector and Its Properties—Marginal and Conditional d.f.’s and p.d.f.’s 112 4.3 Quantiles and Modes of a Distribution 120 4.4* Justification of Statements 1 and 2 123 Chapter 5. Moments of Random Variables—Some Moment and Probability Inequalities 127 5.1 Moments of Random Variables 127 5.2 Expectations and Variances of Some r.v.’s 135 5.3 Conditional Moments of Random Variables 143 5.4 Some Important Applications: Probability and Moment Inequalities 146 5.5 Covariance, Correlation Coefficient and Its Interpretation 150 5.6* Justification of Relation (2) in Chapter 2 155 Chapter 6. Characteristic Functions, Moment Generating Functions and Related Theorems 159 6.1 Preliminaries 159 6.2 Definitions and Basic Theorems—The One-Dimensional Case 161 6.3 The Characteristic Functions of Some Random Variables 167 6.4 Definitions and Basic Theorems—The Multidimensional Case 171 6.5 The Moment Generating Function and Factorial Moment Generating Function of a Random Variable 174 Chapter 7. Stochastic Independence with Some Applications 185 7.1 Stochastic Independence: Criteria of Independence 185 7.2 Proof of Lemma 2 and Related Results 191 7.3 Some Consequences of Independence 194 7.4* Independence of Classes of Events and Related Results 198 Chapter 8. Basic Limit Theorems 201 8.1 Some Modes of Convergence 201 8.2 Relationships Among the Various Modes of Convergence 203 8.3 The Central Limit Theorem 208 8.4 Laws of Large Numbers 217 8.5 Further Limit Theorems 220 8.6* Pólya’s Lemma and Alternative Proof of the WLLN 227 Chapter 9. Transformations of Random Variables and Random Vectors 233 9.1 The Univariate Case 233 9.2 The Multivariate Case 240 9.3 Linear Transformations of Random Vectors 256 9.4 The Probability Integral Transform 263 Chapter 10. Order Statistics and Related Theorems 266 10.1 Order Statistics and Related Distributions 266 10.2 Further Distribution Theory: Probability of Coverage of a Population Quantile 277 Chapter 11. Sufficiency and Related Theorems 280 11.1 Sufficiency: Definition and Some Basic Results 281 11.2 Completeness 292 11.3 Unbiasedness—Uniqueness 295 11.4 The Exponential Family of p.d.f.’s. One-Dimensional Parameter Case 297 11.5 Some Multiparameter Generalizations 302 Chapter 12. Point Estimation 305 12.1 Introduction 305 12.2 Criteria for Selecting an Estimator: Unbiasedness, Minimum Variance 306 12.3 The Case of Availability of Complete Sufficient Statistics 308 12.4 The Case Where Complete Sufficient Statistics Are Not Available or May Not Exist: Cramér-Rao Inequality 314 12.5 Criteria for Selecting an Estimator: The Maximum Likelihood Principle 323 12.6 Criteria for Selecting an Estimator: The Decision-Theoretic Approach 330 12.7 Finding Bayes Estimators 333 12.8 Finding Minimax Estimators 339 12.9 Other Methods of Estimation 341 12.10 Asymptotically Optimal Properties of Estimators 343 12.11 Closing Remarks 346 Chapter 13. Testing Hypotheses 348 13.1 General Concepts of the Neyman–Pearson Testing Hypotheses Theory 348 13.2 Testing a Simple Hypothesis Against a Simple Alternative 350 13.3 UMP Tests for Testing Certain Composite Hypotheses 358 13.4 UMPU Tests for Testing Certain Composite Hypotheses 370 13.5 Testing the Parameters of a Normal Distribution 374 13.6 Comparing the Parameters of Two Normal Distributions 378 13.7 Likelihood Ratio Tests 382 13.8 Applications of LR Tests: Contingency Tables, Goodness-of-Fit Tests 391 13.9 Decision-Theoretic Viewpoint of Testing Hypotheses 396 Chapter 14. Sequential Procedures 403 14.1 Some Basic Theorems of Sequential Sampling 403 14.2 Sequential Probability Ratio Test 409 14.3 Optimality of the SPRT-Expected Sample Size 414 14.4 Some Examples 415 Chapter 15. Confidence Regions—Tolerance Intervals 418 15.1 Confidence Intervals 418 15.2 Some Examples 419 15.3 Confidence Intervals in the Presence of Nuisance Parameters 428 15.4 Confidence Regions—Approximate Confidence Intervals 431 15.5 Tolerance Intervals 434 Chapter 16. The General Linear Hypothesis 437 16.1 Introduction of the Model 437 16.2 Least Square Estimators—Normal Equations 439 16.3 Canonical Reduction of the Linear Model—Estimation of σ 445 16.4 Testing Hypotheses About η= E(Y) 450 16.5 Derivation of the Distribution of the F Statistic 454 Chapter 17. Analysis of Variance 461 17.1 One-way Layout (or One-way Classification) with the Same Number of Observations Per Cell 461 17.2 Two-way Layout (Classification) with One Observation Per Cell 467 17.3 Two-way Layout (Classification) with K (> 2) Observations Per Cell 473 17.4 A Multicomparison method 479 Chapter 18. The Multivariate Normal Distribution 484 18.1 Introduction 484 18.2 Some Properties of Multivariate Normal Distributions 488 18.3 Estimation of μ and Σ and a Test of Independence 490 Chapter 19. Quadratic Forms 497 19.1 Introduction 497 19.2 Some Theorems on Quadratic Forms 498 Chapter 20. Nonparametric Inference 506 20.1 Nonparametric Estimation 506 20.2 Nonparametric Estimation of a p.d.f. 508 20.3 Some Nonparametric Tests 511 20.4 More About Nonparametric Tests: Rank Tests 514 20.5 Sign Test 517 20.6 Relative Asymptotic Efficiency of Tests 518 Appendix I. Topics from Vector and Matrix Algebra 520 I.1 Basic Definitions in Vector Spaces 520 I.2 Some Theorems on Vector Spaces 522 I.3 Basic Definitions About Matrices 523 I.4 Some Theorems About Matrices and Quadratic Forms 525 Appendix II. Noncentral t, X2 and F-Distributions 529 II.1 Noncentral t-Distribution 529 II.2 Noncentral X2-Distribution 529 II.3 Noncentral F-Distribution 530 Appendix III. Tables 532 1 The Cumulative Binomial Distribution 532 2 The Cumulative Poisson Distribution 541 3 The Normal Distribution 544 4 Critical Values for Student’s t-Distribution 547 5 Critical Values for the Chi-Square Distribution 550 6 Critical Values for the F-Distribution 553 7 Table of Selected Discrete and Continuous Distributions and Some of Their Characteristics 563 Some Notation and Abbreviations 566 Answers to Selected Exercises 568 Index 582 This rigorous graduate level textbook is a revised and updated second edition of the 1973 text. The coverage is extensive, with chapters focused on areas from sequential analysis to linear models and much more. Exercises are included at the end of each section, allowing students to implement the learned material quickly and easily, without having to search the whole text for information.