The Theory of Information and Coding (Encyclopedia of Mathematics and its Applications No. 86)

An excellent update of a classic text. This book, in just this one volume, gives you an incisive description of information theory. It assumes that you have no prior experience in this field. It develops the theory from the first principles of Claude Shannon, and rapidly shows you his major results. If you are a student, a valuable and essential part of the book are the several hundred questions. You really need to tackle as many of these as you can. By doing so, you can substantially deepen your understanding of the subject. The problem sets are probably also another reason why this book has become a common text in Information Theory classes. The first edition of this book (and now hopefully this edition!) has been compared by some to Richard Feynman's Lectures on Physics, as a standard and authoritative book in its field. This Is A Revised Edition Of Mceliece's Classic. It Is A Self-contained Introduction To All Basic Results In The Theory Of Information And Coding (invented By Claude Shannon In 1948). This Theory Was Developed To Deal With The Fundamental Problem Of Communication, That Of Reproducing At One Point, Either Exactly Or Approximately, A Message Selected At Another Point. There Is A Short And Elementary Overview Introducing The Reader To The Concept Of Coding. Then, Following The Main Results, The Channel And Source Coding Theorems, There Is A Study Of Specific Coding Schemes Which Can Be Used For Channel And Source Coding. This Volume Can Be Used Either For Self-study, Or For A Graduate/undergraduate Level Course At University. It Includes Dozens Of Worked Examples And Several Hundred Problems For Solution. The Exposition Will Be Easily Comprehensible To Readers With Some Prior Knowledge Of Probability And Linear Algebra. Part 1 Information Theory -- 1 Entropy And Mutual Information 17 -- 1.1 Discrete Random Variables 17 -- 1.2 Discrete Random Vectors 33 -- 1.3 Nondiscrete Random Variables And Vectors 37 -- 2 Discrete Memoryless Channels And Their Capacity-cost Functions 50 -- 2.1 The Capacity-cost Function 50 -- 2.2 The Channel Coding Theorem 58 -- 3 Discrete Memoryless Sources And Their Rate-distortion Functions 75 -- 3.1 The Rate-distortion Function 75 -- 3.2 The Source Coding Theorem 84 -- 4 The Gaussian Channel And Source 95 -- 4.1 The Gaussian Channel 95 -- 4.2 The Gaussian Source 99 -- 5 The Source-channel Coding Theorem 112 -- 6 Survey Of Advanced Topics For Part One 123 -- 6.2 The Channel Coding Theorem 123 -- 6.3 The Source Coding Theorem 131 -- Part 2 Coding Theory -- 7 Linear Codes 139 -- 7.1 Introduction: The Generator And Parity-check Matrices 139 -- 7.2 Syndrome Decoding On Q-ary Symmetric Channels 143 -- 7.3 Hamming Geometry And Code Performance 146 -- 7.4 Hamming Codes 148 -- 7.5 Syndrome Decoding On General Q-ary Channels 149 -- 7.6 Weight Enumerators And The Macwilliams Identities 153 -- 8 Cyclic Codes 167 -- 8.2 Shift-register Encoders For Cyclic Codes 181 -- 8.3 Cyclic Hamming Codes 195 -- 8.4 Burst-error Correction 199 -- 8.5 Decoding Burst-error Correcting Cyclic Codes 215 -- 9 Bch, Reed-solomon, And Related Codes 230 -- 9.2 Bch Codes As Cyclic Codes 234 -- 9.3 Decoding Bch Codes, Part One: The Key Equation 236 -- 9.4 Euclid's Algorithm For Polynomials 244 -- 9.5 Decoding Bch Codes, Part Two: The Algorithms 249 -- 9.6 Reed-solomon Codes 253 -- 9.7 Decoding When Erasures Are Present 266 -- 9.8 The (23,12) Golay Code 277 -- 10 Convolutional Codes 293 -- 10.2 State Diagrams, Trellises, And Viterbi Decoding 300 -- 10.3 Path Enumerators And Error Bounds 307 -- 10.4 Sequential Decoding 313 -- 11 Variable-length Source Coding 330 -- 11.2 Uniquely Decodable Variable-length Codes 331 -- 11.3 Matching Codes To Sources 334 -- 11.4 The Construction Of Optimal Ud Codes (huffman's Algorithm) 337 -- 12 Survey Of Advanced Topics For Part Two 347 -- 12.2 Block Codes 347 -- 12.3 Convolutional Codes 357 -- 12.4 A Comparison Of Block And Convolutional Codes 359 -- 12.5 Source Codes 363 -- A Probability Theory 366 -- B Convex Functions And Jensen's Inequality 370 -- C Finite Fields 375 -- D Path Enumeration In Directed Graphs 380 -- 1 General Reference Textbooks 384 -- 2 An Annotated Bibliography Of The Theory Of Information And Coding 384. R.j. Mceliece. Previous Ed.: Reading, Mass. : Addison-wesley, 1977. Includes Bibliographical References And Index. "This volume is a self-contained introduction to all basic results in the theory of information and coding (invented by Claude Shannon in 1948). This theory was developed to deal with the fundamental problem of communication, that of reproducing at one point, either exactly or approximately, a message selected at another point. There is a short and elementary overview introducing the reader to the concept of coding. Then, following the main results, the channel and source coding theorems, there is a study of specific coding schemes which can be used for channel and source coding. This volume can be used either for self-study, or for a graduate/undergraduate level course at university. It includes dozens of worked examples and several hundred problems for solution. The exposition will be easily comprehensible to readers with some prior knowledge of probability and linear algebra." -- BOOK JACKET This is a self-contained introduction to the theory of information and coding. It can be used either for self-study or as the basis for a course at either the graduate or, undergraduate level. The text includes dozens of worked examples and several hundred problems for solution.