Formal Theories of Information: From Shannon to Semantic Information Theory and General Concepts of Information (Lecture Notes in Computer Science (5363))

This book presents the scientific outcome of a joint effort of the computer science departments of the universities of Berne, Fribourg and Neuchâtel. Within an initiative devoted to "Information and Knowledge", these research groups collaborated over several years on issues of logic, probability, inference, and deduction. The goal of this volume is to examine whether there is any common ground between the different approaches to the concept of information. The structure of this book could be represented by a circular model, with an innermost syntactical circle, comprising statistical and algorithmic approaches; a second, larger circle, the semantical one, in which "meaning" enters the stage; and finally an outermost circle, the pragmatic one, casting light on real-life logical reasoning. These articles are complemented by two philosophical contributions exploring the wide conceptual field as well as taking stock of the articles on the various formal theories of information. It is commonly assumed that computers process information. But what is information? In a technical, important, but nevertheless rather narrow sense, Shanons information theory gives a first answer to this question. This theory focuses on measuring the information content of a message. Essentially this measure is the reduction of the uncertainty obtained by receiving a message. The uncertainty of a situation of ignorance in turn is measured by entropy. This theory has had an immense impact on the technology of information storage, data compression, information transmission and coding and still is a very active domain of research. Shannons theory has also attracted much interest in a more philosophic look at information, although it was readily remarked that it is only a syntactic theory of information and neglects semantic issues. Several attempts have been made in philosophy to give information theory a semantic favor, but still mostly based on or at least linked to Shannons theory. Approaches to semantic information theory also very often make use of formal logic. Thereby, information is linked to reasoning, deduction and inference, as well as to decision making. Further, entropy and related measure were soon found to have important connotations with regard to statistical inference. Surely, statistical data and observation represent information, information about unknown, hidden parameters. Thus a whole branch of statistics developed around concepts of Shannons information theory or derived from them. Also some proper measurements - propriate for statistics, like Fishers information, were proposed. It Is Commonly Assumed That Computers Process Information. But What Is Inf- Mation? In A Technical, Important, But Nevertheless Rather Narrow Sense, Sh- Non’sinformationtheorygivesa?rstanswertothisquestion.thistheoryfocuses On Measuring The Information Content Of A Message. Essentially This Measure Is The Reduction Of The Uncertainty Obtained By Receiving A Message. The Unc- Tainty Of A Situation Of Ignorance In Turn Is Measured By Entropy. This Theory Hashad An Immense Impact On The Technologyof Information Storage,data C- Pression, Information Transmission And Coding And Still Is A Very Active Domain Of Research. Shannon’s Theory Has Also Attractedmuch Interest In A More Philosophic Look At Information, Although It Was Readily Remarked That It Is Only A “syntactic” Theory Of Information And Neglects “semantic” Issues. Several Attempts Have Been Made In Philosophy To Give Information Theory A Semantic ?avor, But Still Mostly Based On Or At Least Linked To Shannon’s Theory. Approaches To Semantic Informationtheoryalsoveryoftenmakeuseofformallogic.thereby,information Is Linked To Reasoning, Deduction And Inference, As Well As To Decision Making. Further, Entropy And Related Measure Were Soon Found To Have Important Connotations With Regard To Statistical Inference. Surely, Statistical Data And Observation Represent Information, Information About Unknown, Hidden Para- Ters. Thus A Whole Branch Of Statistics Developed Around Concepts Of Shannon’s Information Theory Or Derived From Them. Also Some Proper Measurements - Propriate For Statistics, Like Fisher’s Information, Were Proposed. Front Matter....Pages - Introduction....Pages 1-12 Philosophical Conceptions of Information....Pages 13-53 Information Theory, Relative Entropy and Statistics....Pages 54-78 Information: The Algorithmic Paradigm....Pages 79-94 Information Algebra....Pages 95-127 Uncertain Information....Pages 128-160 Comparing Questions and Answers: A Bit of Logic, a Bit of Language, and Some Bits of Information....Pages 161-192 Channels: From Logic to Probability....Pages 193-233 Modeling Real Reasoning....Pages 234-252 One or Many Concepts of Information?....Pages 253-267 Back Matter....Pages - "This book presents the scientific outcome of a joint effort of the computer science departments of the universities of Berne, Fribourg and Neuchâtel. Within an initiative devoted to 'Information and Knowledge', these research groups collaborated over several years on issues of logic, probability, inference, and deduction. The goal of this volume is to examine whether there is any common ground between the different approaches to the concept of information."-- Page 4 of cover