Adaptive Languages: An Information-theoretic Account of Linguistic Diversity (Trends in Linguistics. Studies and Monographs) (Trends in Linguistics. Studies and Monographs [tilsm], 316)

Languages carry information. To fulfil this purpose, they employ a multitude of coding strategies. This book explores a core property of linguistic coding – called lexical diversity. Parallel text corpora of overall more than 1800 texts written in more than 1200 languages are the basis for computational analyses. Different measures of lexical diversity are discussed and tested, and Shannon’s measure of uncertainty – the entropy – is chosen to assess differences in the distributions of words. To further explain this variation, a range of descriptive, explanatory, and grouping factors are considered in a series of statistical models. The first category includes writing systems, word-formation patterns, registers and styles. The second category includes population size, non-native speaker proportions and language status. Grouping factors further elicit whether the results extrapolate across – or are limited to – specific language families and areas. This account marries information-theoretic methods with a complex systems framework, illustrating how languages adapt to the varying needs of their users. It sheds light on the puzzling diversity of human languages in a quantitative, data driven and reproducible manner. Preface 7 Contents 9 Abbreviations 12 List of Tables 13 List of Figures 14 1. Introduction 17 2. Languages as Adaptive Systems 25 3. Language Change and Population Structure 43 4. Lexical Diversity across Languages of the World 58 5. Descriptive Factors: Language “Internal” Effects 94 6. Explanatory Factors: Language “External” Effects 127 7. Grouping Factors: Language Families and Areas 137 8. Predicting Lexical Diversity: Statistical Models 146 9. Explaining Diversity: Multiple Factors Interacting 156 10. Further Problems and Caveats 172 11. Conclusions: Universality and Diversity 199 12. Appendix A: Advanced Entropy Estimators 202 13. Appendix B: Multiple Regression Assumptions 205 14. Appendix C: Mixed-effects Regression Assumptions 208 Bibliography 211 Index 231 "Languages transmit information. They are vessels carrying meaning across metres, kilometres, and around the globe. This book harnesses information theory to measure word-level differences in more than 1200 languages. It further models geographic, demographic and social factors to explain the diversity of encoding strategies. The results support the framework of languages as complex adaptive systems which are shaped by the varying needs of their users." -- Page 4 of cover