FRONTIERS IN STATISTICS: DEDICATED TO PETER JOHN BICKEL IN HONOR OF HIS 65TH BIRTHDAY; ED. BY JIANQING FAN

During the last two decades, many areas of statistical inference have experienced phenomenal growth. This book presents a timely analysis and overview of some of these new developments and a contemporary outlook on the various frontiers of statistics. Eminent leaders in the field have contributed 16 review articles and 6 research articles covering areas including semi-parametric models, data analytical nonparametric methods, statistical learning, network tomography, longitudinal data analysis, financial econometrics, time series, bootstrap and other re-sampling methodologies, statistical computing, generalized nonlinear regression and mixed effects models, martingale transform tests for model diagnostics, robust multivariate analysis, single index models and wavelets. This volume is dedicated to Prof. Peter J Bickel in honor of his 65th birthday. The first article of this volume summarizes some of Prof. Bickel's distinguished contributions. Contents......Page 12 1.1 Introduction......Page 29 1.2 Doing Well at a Point and Beyond......Page 30 1.4 Distribution Free Tests Higher Order Expansions and Challenging Projects......Page 32 1.5 From Adaptive Estimation to Semiparametric Models......Page 33 1.6 Hidden Markov Models......Page 34 1.7 Non- and Semi-parametric Testing......Page 35 References......Page 36 Bickel's Publication......Page 39 Part I. Semiparametric Modeling......Page 51 2.1 Introduction......Page 53 2.3 Testing and Profile Likelihood Theory......Page 56 2.4 Semiparametric Mixture Model Theory......Page 57 2.6 Bayes Methods and Theory......Page 58 2.7 Model Selection Methods......Page 59 2.9 Transformation and Frailty Models......Page 60 2.10 Semiparametric Regression Models......Page 61 2.11 Extensions to Non-i.i.d. Data......Page 62 2.12 Critiques and Possible Alternative Theories......Page 63 References......Page 64 3.1 Introduction......Page 73 3.2 Characterization of Efficient Estimators......Page 75 3.3 Autoregression Parameter......Page 78 3.4 Innovation Distribution......Page 80 3.5 Innovation Density......Page 82 3.6 Conditional Expectation......Page 83 3.7 Stationary Distribution......Page 85 3.8 Stationary Density......Page 86 3.9 Transition Density......Page 87 References......Page 88 4.1 Introduction......Page 91 4.2 Estimation via Outer Product of Gradients......Page 94 4.3 Global Minimization Estimation Methods......Page 96 4.4 Sliced Inverse Regression Method......Page 98 4.5 Asymptotic Distributions......Page 99 4.6 Comparisons in Some Special Cases......Page 101 4.7 Proofs of the Theorems......Page 102 References......Page 112 5.1 Introduction......Page 115 5.2 Estimating Function Based Cross-Validation......Page 118 5.3 Some Examples......Page 123 5.4 General Finite Sample Result......Page 129 5.5 Appendix......Page 133 References......Page 136 Part II. Nonparametric Methods......Page 139 6. Powerful Choices: Tuning Parameter Selection Based on Power......Page 141 6.1 Introduction: Local Testing and Asymptotic Power......Page 142 6.2 Maximizing Asymptotic Power......Page 144 6.3 Examples......Page 157 6.4 Appendix......Page 162 References......Page 167 7. Nonparametric Assessment of Atypicality......Page 171 7.1 Introduction......Page 172 7.2 Estimating Atypicality......Page 173 7.3 Theoretical Properties......Page 176 7.4 Numerical Properties......Page 179 7.5 Outline of Proof of Theorem 7.1......Page 185 References......Page 188 8.1 Introduction......Page 191 8.2 Wavelets......Page 192 8.3 Nonparametric Regression......Page 194 8.4 Inverse Problems......Page 200 8.5 Change-points......Page 202 8.6 Local Self-similarity and Non-stationary Stochastic Process......Page 204 References......Page 207 9.1 Introduction......Page 211 9.2 Lack-of-fit Tests......Page 225 9.3 Censoring......Page 229 9.4 Khamaladze Transform or Bootstrap......Page 230 References......Page 231 Part III. Statistical Learning and Bootstrap......Page 235 10.1 Introduction......Page 237 10.2 Boosting and Functional Gradient Descent......Page 239 10.3 L2-Boosting for High-dimensional Multivariate Regression......Page 245 10.4 L2-Boosting for Multivariate Linear Time Series......Page 250 References......Page 257 11.1 Introduction......Page 259 11.2 Bootstrap for i.i.d Data......Page 261 11.3 Model Based Bootstrap......Page 266 11.4 Block Bootstrap......Page 268 11.5 Sieve Bootstrap......Page 271 11.6 Transformation Based Bootstrap......Page 272 11.7 Bootstrap for Markov Processes......Page 273 11.8 Bootstrap under Long Range Dependence......Page 274 11.9 Bootstrap for Spatial Data......Page 276 References......Page 278 12.1 Introduction......Page 285 12.2 Proof of Theorem 12.1......Page 290 12.3 Evaluation of the Oscillatory Term......Page 299 References......Page 301 Part IV. Longitudinal Data Analysis......Page 303 13.1 Introduction......Page 305 13.2 Nonparametric Model with a Single Covariate......Page 307 13.3 Partially Linear Models......Page 311 13.4 Varying-Coefficient Models......Page 319 13.5 An Illustration......Page 321 13.6 Generalizations......Page 322 13.7 Estimation of Covariance Matrix......Page 324 References......Page 327 14.1 Introduction and Review......Page 333 14.2 The Functional Approach to Longitudinal Responses......Page 339 14.3 Predicting Longitudinal Trajectories from a Covariate......Page 341 14.4 Illustrations......Page 344 References......Page 349 Part V. Statistics in Science and Technology......Page 353 15. Statistical Physics and Statistical Computing: A Critical Link......Page 355 15.2 The Ising Model......Page 356 15.3 The Swendsen-Wang Algorithm and Criticality......Page 357 15.4 Instantaneous Hellinger Distance and Heat Capacity......Page 359 15.5 A Brief Overview of Perfect Sampling......Page 362 15.6 Huber's Bounding Chain Algorithm......Page 364 15.7 Approximating Criticality via Coupling Time......Page 368 15.8 A Speculation......Page 370 References......Page 371 16. Network Tomography: A Review and Recent Developments......Page 373 16.1 Introduction......Page 374 16.2 Passive Tomography......Page 376 16.3 Active Tomography......Page 380 16.4 An Application......Page 387 16.5 Concluding Remarks......Page 391 References......Page 392 Part VI. Financial Econometrics......Page 395 17.1 Introduction......Page 397 17.2 The Univariate Case......Page 399 17.3 Multivariate Likelihood Expansions......Page 406 17.4 Connection to Saddlepoint Approximations......Page 411 17.5 An Example with Nonlinear Drift and Diffusion Specifications......Page 414 17.6 An Example with Stochastic Volatility......Page 417 17.7 Inference When the State is Partially Observed......Page 419 17.8 Application to Specification Testing......Page 427 17.9 Derivative Pricing Applications......Page 428 17.10 Likelihood Inference for Diffusions under Nonstationarity......Page 429 References......Page 430 18.1 The Frontier Model......Page 435 18.2 Envelope Estimators......Page 437 18.3 Order-m Estimators......Page 445 18.4 Conditional Frontier Models......Page 449 18.5 Outlook......Page 451 References......Page 452 Part VII. Parametric Techniques and Inferences......Page 455 19.1 Introduction......Page 457 19.2 Newton's Estimate of Mixing Distributions......Page 459 19.3 Review of Newton's Result on Convergence......Page 460 19.4 Convergence Results......Page 461 19.5 Other Results......Page 466 19.6 Simulation......Page 468 References......Page 470 20.1 Introduction......Page 473 20.2 Linear Mixed Models......Page 474 20.3 Generalized Linear Mixed Models......Page 478 20.4 Nonlinear Mixed Effects Models......Page 483 References......Page 488 21.1 Introduction......Page 495 21.2 Robustness Criteria......Page 497 21.3 Robust Multivariate Location and Scatter Estimators......Page 501 21.4 Applications......Page 509 21.5 Conclusions and Future Works......Page 512 References......Page 513 22.1 Introduction......Page 519 22.2 Kullback-Leibler Loss and Exponential Families......Page 521 22.3 Mean Square Error Loss......Page 523 22.4 Location Families......Page 524 22.5 Approximate Solutions......Page 526 22.6 Convergence of the Loss Estimate......Page 530 References......Page 534 Subject Index......Page 535 Author Index......Page 539 Contents 12 1. Our Steps on the Bickel Way 29 1.1 Introduction 29 1.2 Doing Well at a Point and Beyond 30 1.3 Robustness Transformations Oracle-free Inference and Stable Parameters 32 1.4 Distribution Free Tests Higher Order Expansions and Challenging Projects 32 1.5 From Adaptive Estimation to Semiparametric Models 33 1.6 Hidden Markov Models 34 1.7 Non- and Semi-parametric Testing 35 1.8 The Road to Real Life 36 References 36 Bickel's Publication 39 Part I. Semiparametric Modeling 51 2. Semiparametric Models: A Review of Progress since BKRW (1993) 53 2.1 Introduction 53 2.2 Missing Data Models 56 2.3 Testing and Profile Likelihood Theory 56 2.4 Semiparametric Mixture Model Theory 57 2.5 Rates of Convergence via Empirical Process Methods 58 2.6 Bayes Methods and Theory 58 2.7 Model Selection Methods 59 2.8 Empirical Likelihood 60 2.9 Transformation and Frailty Models 60 2.10 Semiparametric Regression Models 61 2.11 Extensions to Non-i.i.d. Data 62 2.12 Critiques and Possible Alternative Theories 63 References 64 3. Efficient Estimator for Time Series 73 3.1 Introduction 73 3.2 Characterization of Efficient Estimators 75 3.3 Autoregression Parameter 78 3.4 Innovation Distribution 80 3.5 Innovation Density 82 3.6 Conditional Expectation 83 3.7 Stationary Distribution 85 3.8 Stationary Density 86 3.9 Transition Density 87 References 88 4. On the Efficiency of Estimation for a Single-index Model 91 4.1 Introduction 91 4.2 Estimation via Outer Product of Gradients 94 4.3 Global Minimization Estimation Methods 96 4.4 Sliced Inverse Regression Method 98 4.5 Asymptotic Distributions 99 4.6 Comparisons in Some Special Cases 101 4.7 Proofs of the Theorems 102 References 112 5. Estimating Function Based Cross-Validation 115 5.1 Introduction 115 5.2 Estimating Function Based Cross-Validation 118 5.3 Some Examples 123 5.4 General Finite Sample Result 129 5.5 Appendix 133 References 136 Part II. Nonparametric Methods 139 6. Powerful Choices: Tuning Parameter Selection Based on Power 141 6.1 Introduction: Local Testing and Asymptotic Power 142 6.2 Maximizing Asymptotic Power 144 6.3 Examples 157 6.4 Appendix 162 References 167 7. Nonparametric Assessment of Atypicality 171 7.1 Introduction 172 7.2 Estimating Atypicality 173 7.3 Theoretical Properties 176 7.4 Numerical Properties 179 7.5 Outline of Proof of Theorem 7.1 185 References 188 8. Selective Review on Wavelets in Statistics 191 8.1 Introduction 191 8.2 Wavelets 192 8.3 Nonparametric Regression 194 8.4 Inverse Problems 200 8.5 Change-points 202 8.6 Local Self-similarity and Non-stationary Stochastic Process 204 8.7 Beyond Wavelets 207 References 207 9. Model Diagnostics via Martingale Transforms: A Brief Review 211 9.1 Introduction 211 9.2 Lack-of-fit Tests 225 9.3 Censoring 229 9.4 Khamaladze Transform or Bootstrap 230 References 231 Part III. Statistical Learning and Bootstrap 235 10. Boosting Algorithms: with an Application to Bootstrapping Multivariate Time Series 237 10.1 Introduction 237 10.2 Boosting and Functional Gradient Descent 239 10.3 L2-Boosting for High-dimensional Multivariate Regression 245 10.4 L2-Boosting for Multivariate Linear Time Series 250 References 257 11. Bootstrap Methods: A Review 259 11.1 Introduction 259 11.2 Bootstrap for i.i.d Data 261 11.3 Model Based Bootstrap 266 11.4 Block Bootstrap 268 11.5 Sieve Bootstrap 271 11.6 Transformation Based Bootstrap 272 11.7 Bootstrap for Markov Processes 273 11.8 Bootstrap under Long Range Dependence 274 11.9 Bootstrap for Spatial Data 276 References 278 12. An Expansion for a Discrete Non-Lattice Distribution 285 12.1 Introduction 285 12.2 Proof of Theorem 12.1 290 12.3 Evaluation of the Oscillatory Term 299 References 301 Part IV. Longitudinal Data Analysis 303 13. An Overview on Nonparametric and Semiparametric Techniques for Longitudinal Data 305 13.1 Introduction 305 13.2 Nonparametric Model with a Single Covariate 307 13.3 Partially Linear Models 311 13.4 Varying-Coefficient Models 319 13.5 An Illustration 321 13.6 Generalizations 322 13.7 Estimation of Covariance Matrix 324 References 327 14. Regressing Longitudinal Response Trajectories on a Covariate 333 14.1 Introduction and Review 333 14.2 The Functional Approach to Longitudinal Responses 339 14.3 Predicting Longitudinal Trajectories from a Covariate 341 14.4 Illustrations 344 References 349 Part V. Statistics in Science and Technology 353 15. Statistical Physics and Statistical Computing: A Critical Link 355 15.1 MCMC Revolution and Cross-Fertilization 356 15.2 The Ising Model 356 15.3 The Swendsen-Wang Algorithm and Criticality 357 15.4 Instantaneous Hellinger Distance and Heat Capacity 359 15.5 A Brief Overview of Perfect Sampling 362 15.6 Huber's Bounding Chain Algorithm 364 15.7 Approximating Criticality via Coupling Time 368 15.8 A Speculation 370 References 371 16. Network Tomography: A Review and Recent Developments 373 16.1 Introduction 374 16.2 Passive Tomography 376 16.3 Active Tomography 380 16.4 An Application 387 16.5 Concluding Remarks 391 References 392 Part VI. Financial Econometrics 395 17. Likelihood Inference for Diffusions: A Survey 397 17.1 Introduction 397 17.2 The Univariate Case 399 17.3 Multivariate Likelihood Expansions 406 17.4 Connection to Saddlepoint Approximations 411 17.5 An Example with Nonlinear Drift and Diffusion Specifications 414 17.6 An Example with Stochastic Volatility 417 17.7 Inference When the State is Partially Observed 419 17.8 Application to Specification Testing 427 17.9 Derivative Pricing Applications 428 17.10 Likelihood Inference for Diffusions under Nonstationarity 429 References 430 18. Nonparametric Estimation of Production Efficiency 435 18.1 The Frontier Model 435 18.2 Envelope Estimators 437 18.3 Order-m Estimators 445 18.4 Conditional Frontier Models 449 18.5 Outlook 451 References 452 Part VII. Parametric Techniques and Inferences 455 19. Convergence and Consistency of Newton's Algorithm for Estimating Mixing Distribution 457 19.1 Introduction 457 19.2 Newton's Estimate of Mixing Distributions 459 19.3 Review of Newton's Result on Convergence 460 19.4 Convergence Results 461 19.5 Other Results 466 19.6 Simulation 468 References 470 20. Mixed Models: An Overview 473 20.1 Introduction 473 20.2 Linear Mixed Models 474 20.3 Generalized Linear Mixed Models 478 20.4 Nonlinear Mixed Effects Models 483 References 488 21. Robust Location and Scatter Estimators in Multivariate Analysis 495 21.1 Introduction 495 21.2 Robustness Criteria 497 21.3 Robust Multivariate Location and Scatter Estimators 501 21.4 Applications 509 21.5 Conclusions and Future Works 512 References 513 22. Estimation of the Loss of an Estimate 519 22.1 Introduction 519 22.2 Kullback-Leibler Loss and Exponential Families 521 22.3 Mean Square Error Loss 523 22.4 Location Families 524 22.5 Approximate Solutions 526 22.6 Convergence of the Loss Estimate 530 References 534 Subject Index 535 Author Index 539 Presents an analysis and overview of some of the developments in the various frontiers of statistics. This book includes articles covering areas including semi-parametric models, data analytical nonparametric methods, statistical learning, network tomography, longitudinal data analysis, financial econometrics, statistical computing, and more.