Mathematics of Information: Theory and Applications of Shannon-Wiener Information

Starting with the Shannon-Wiener approach to mathematical information theory, allowing a mathematical "measurement" of an amount of information, the book begins by defining the terms message and information and axiomatically assigning an amount of information to a probability. The second part explores countable probability spaces, leading to the definition of Shannon entropy based on the average amount of information; three classical applications of Shannon entropy in statistical physics, mathematical statistics, and communication engineering are presented, along with an initial glimpse into the field of quantum information. The third part is dedicated to general probability spaces, focusing on the information-theoretical analysis of dynamic systems. The book builds on bachelor-level knowledge and is primarily intended for mathematicians and computer scientists, placing a strong emphasis on rigorous proofs. Introduction Contents Symbols List of Figures List of Tables Part I Basics 1 Message and Information 1.1 Starting Point Transmitter: Message 1.2 Endpoint Receiver: Information 2 Information and Chance 2.1 Probability and Amount of Information 2.2 The Average Information Quantity of a Character Part II Countable Systems 3 The Entropy 3.1 Discrete Probability Spaces 3.2 Average Amount of Information 3.3 Huffman Coding 4 The Maximum Entropy Principle 4.1 Maximum Average Information under Constraints 4.2 Statistical Physics 5 Conditional Probabilities 5.1 Sufficiency 5.2 Transinformation 6 Quantum Information 6.1 Q-Bits 6.2 Tensor Spaces and Multi-Q-Bits 6.3 Measurements 6.4 Copying Part III General Systems 7 The Entropy of Partitions 7.1 Uncountable Outcomes 7.2 Entropy 7.3 Entropy in Dynamic Systems 8 Stationary Information Sources 8.1 Cylinder Sets and Projections 8.2 Entropy per Character 9 Density Functions and Entropy 9.1 Integration 9.2 Densities 9.3 Differential Entropy 9.4 Differential Entropy in Dynamic Systems 10 Conditional Expectations 10.1 Existence and Uniqueness 10.2 Sufficiency References Information Theory as a Mathematical Subdiscipline and Interface to Computer ScienceExact Anchoring of the Concept of Information in Probability TheoryIntroduces Applications of Information Theory, Including Mathematical Statistics, Statistical Physics, and Communication Engineering