Maximum Likelihood Estimation with Stata, Fourth Edition

Maximum Likelihood Estimation with Stata, Fourth Edition is written for researchers in all disciplines who need to compute maximum likelihood estimators that are not available as prepackaged routines. Readers are presumed to be familiar with Stata, but no special programming skills are assumed except in the last few chapters, which detail how to add a new estimation command to Stata. The book begins with an introduction to the theory of maximum likelihood estimation with particular attention on the practical implications for applied work. Individual chapters then describe in detail each of the four types of likelihood evaluator programs and provide numerous examples, such as logit and probit regression, Weibull regression, random-effects linear regression, and the Cox proportional hazards model. Later chapters and appendixes provide additional details about the ml command, provide checklists to follow when writing evaluators, and show how to write your own estimation commands. Stata and Statistics List of tables 13 List of figures 13 Preface to the fourth edition 13 Versions of Stata 13 Notation and typography 13 Theory and practice 23 The likelihood-maximization problem 24 Likelihood theory 26 All results are asymptotic 30 Likelihood-ratio tests and Wald tests 31 The outer product of gradients variance estimator 32 Robust variance estimates 33 The maximization problem 35 Numerical root finding 35 Newton's method 35 The Newton--Raphson algorithm 37 Quasi-Newton methods 39 The BHHH algorithm 40 The DFP and BFGS algorithms 40 Numerical maximization 41 Numerical derivatives 42 Numerical second derivatives 46 Monitoring convergence 47 Introduction to ml 51 The probit model 51 Normal linear regression 54 Robust standard errors 56 Weighted estimation 57 Other features of method-gf0 evaluators 58 Limitations 58 Overview of ml 61 The terminology of ml 61 Equations in ml 62 Likelihood-evaluator methods 70 Tools for the ml programmer 73 Common ml options 73 Subsamples 73 Weights 74 OPG estimates of variance 75 Robust estimates of variance 76 Survey data 78 Constraints 79 Choosing among the optimization algorithms 79 Maximizing your own likelihood functions 83 Method lf 85 The linear-form restrictions 86 Examples 87 The probit model 87 Normal linear regression 88 The Weibull model 91 The importance of generating temporary variables as doubles 93 Problems you can safely ignore 95 Nonlinear specifications 96 The advantages of lf in terms of execution speed 97 Methods lf0, lf1, and lf2 99 Comparing these methods 99 Outline of evaluators of methods lf0, lf1, and lf2 100 The todo argument 101 The b argument 101 Using mleval to obtain values from each equation 102 The lnfj argument 104 Arguments for scores 105 The H argument 106 Using mlmatsum to define H 108 Aside: Stata's scalars 109 Summary of methods lf0, lf1, and lf2 112 Method lf0 112 Method lf1 114 Method lf2 116 Examples 118 The probit model 118 Normal linear regression 120 The Weibull model 126 Methods d0, d1, and d2 131 Comparing these methods 131 Outline of method d0, d1, and d2 evaluators 132 The todo argument 133 The b argument 133 The lnf argument 134 Using lnf to indicate that the likelihood cannot be calculated 135 Using mlsum to define lnf 136 The g argument 138 Using mlvecsum to define g 138 The H argument 140 Summary of methods d0, d1, and d2 141 Method d0 141 Method d1 144 Method d2 146 Panel-data likelihoods 148 Calculating lnf 150 Calculating g 154 Calculating H 158 Using mlmatbysum to help define H 158 Other models that do not meet the linear-form restrictions 166 Debugging likelihood evaluators 173 ml check 173 Using the debug methods 175 First derivatives 177 Second derivatives 187 ml trace 190 Setting initial values 193 ml search 194 ml plot 197 ml init 199 Interactive maximization 203 The iteration log 203 Pressing the Break key 204 Maximizing difficult likelihood functions 206 Final results 209 Graphing convergence 209 Redisplaying output 210 Mata-based likelihood evaluators 215 Introductory examples 215 The probit model 215 The Weibull model 218 Evaluator function prototypes 220 Method-lf evaluators 221 lf-family evaluators 221 d-family evaluators 222 Utilities 223 Dependent variables 224 Obtaining model parameters 224 Summing individual or group-level log likelihoods 225 Calculating the gradient vector 225 Calculating the Hessian 226 Random-effects linear regression 227 Calculating lnf 228 Calculating g 229 Calculating H 230 Results at last 231 Writing do-files to maximize likelihoods 235 The structure of a do-file 235 Putting the do-file into production 236 Writing ado-files to maximize likelihoods 239 Writing estimation commands 239 The standard estimation-command outline 241 Outline for estimation commands using ml 242 Using ml in noninteractive mode 243 Advice 244 Syntax 245 Estimation subsample 247 Parsing with help from mlopts 251 Weights 254 Constant-only model 255 Initial values 259 Saving results in e() 262 Displaying ancillary parameters 262 Exponentiated coefficients 264 Offsetting linear equations 266 Program properties 268 Writing ado-files for survey data analysis 271 Program properties 271 Writing your own predict command 274 Other examples 277 The logit model 277 The probit model 279 Normal linear regression 281 The Weibull model 284 The Cox proportional hazards model 287 The random-effects regression model 290 The seemingly unrelated regression model 293 Syntax of ml 307 Likelihood-evaluator checklists 329 Method lf 329 Method d0 330 Method d1 331 Method d2 333 Method lf0 336 Method lf1 337 Method lf2 339 Listing of estimation commands 343 The logit model 343 The probit model 345 The normal model 347 The Weibull model 349 The Cox proportional hazards model 352 The random-effects regression model 354 The seemingly unrelated regression model 357 References 357 Author index 357 Subject index 357 Maximum Likelihood Estimation with Stata, Fourth Edition is the essential reference and guide for researchers in all disciplines who wish to write maximum likelihood (ML) estimators in Stata. Beyond providing comprehensive coverage of Stata's ml command for writing ML estimators, the book presents an overview of the underpinnings of maximum likelihood and how to think about ML estimation. The book shows you how to take full advantage of the ml command's noteworthy features: - linear constraints - four optimization algorithms (Newton–Raphson, DFP, BFGS, and BHHH) - observed information matrix (OIM) variance estimator - outer product of gradients (OPG) variance estimator - Huber/White/sandwich robust variance estimator - cluster–robust variance estimator - complete and automatic support for survey data analysis - direct support of evaluator functions written in Mata When appropriate options are used, many of these features are provided automatically by ml and require no special programming or intervention by the researcher writing the estimator. The fourth edition has been updated to include new features introduced in recent versions of Stata. Such features include new methods for handling scores, more consistent arguments for likelihood-evaluator programs, and support for likelihood evaluators written in Mata (Stata's matrix programming language). The authors illustrate how to write your estimation command so that it fully supports factor-variable notation and the svy prefix for estimation with survey data. They have also restructured the chapters that introduce ml in a way that allows you to begin working with ml faster. This edition is essential for anyone using Stata 11. In the final chapter, the authors illustrate the major steps required to get from log-likelihood function to fully operational estimation command. This is done using several different models: logit and probit, linear regression, Weibull regression, the Cox proportional hazards model, random-effects regression, and seemingly unrelated regression. The authors provide extensive advice for developing your own estimation commands. With a little care and the help of this book, users will be able to write their own estimation commands—commands that look and behave just like the official estimation commands in Stata. Whether you want to fit a special ML estimator for your own research or wish to write a general-purpose ML estimator for others to use, you need this book.