Mathematics of Big Data: Spreadsheets, Databases, Matrices, and Graphs (MIT Lincoln Laboratory Series)

The First Book To Present The Common Mathematical Foundations Of Big Data Analysis Across A Range Of Applications And Technologies. Today, The Volume, Velocity, And Variety Of Data Are Increasing Rapidly Across A Range Of Fields, Including Internet Search, Healthcare, Finance, Social Media, Wireless Devices, And Cybersecurity. Indeed, These Data Are Growing At A Rate Beyond Our Capacity To Analyze Them. The Tools--including Spreadsheets, Databases, Matrices, And Graphs--developed To Address This Challenge All Reflect The Need To Store And Operate On Data As Whole Sets Rather Than As Individual Elements. This Book Presents The Common Mathematical Foundations Of These Data Sets That Apply Across Many Applications And Technologies. Associative Arrays Unify And Simplify Data, Allowing Readers To Look Past The Differences Among The Various Tools And Leverage Their Mathematical Similarities In Order To Solve The Hardest Big Data Challenges. The Book First Introduces The Concept Of The Associative Array In Practical Terms, Presents The Associative Array Manipulation System D4m (dynamic Distributed Dimensional Data Model), And Describes The Application Of Associative Arrays To Graph Analysis And Machine Learning. It Provides A Mathematically Rigorous Definition Of Associative Arrays And Describes The Properties Of Associative Arrays That Arise From This Definition. Finally, The Book Shows How Concepts Of Linearity Can Be Extended To Encompass Associative Arrays. Mathematics Of Big Data Can Be Used As A Textbook Or Reference By Engineers, Scientists, Mathematicians, Computer Scientists, And Software Engineers Who Analyze Big Data. Preface About the Authors About the Cover Acknowledgments Applications and Practice Introduction and Overview Mathematics of Data Data in the World Mathematical Foundations Making Data Rigorous Conclusions, Exercises, and References Perspectives on Data Interrelations Spreadsheets Databases Matrices Graphs Map Reduce Other Perspectives Conclusions, Exercises, and References Dynamic Distributed Dimensional Data Model Background Design Matrix Mathematics Common SQL, NoSQL, NewSQL Interface Key-Value Store Database Schema Data-Independent Analytics Parallel Performance Computing on Masked Data Conclusions, Exercises, and References Associative Arrays and Musical Metadata Data and Metadata Dense Data Dense Operations Sparse Data Sparse Operations Conclusions, Exercises, and References Associative Arrays and Abstract Art Visual Abstraction Minimal Adjacency Array Symmetric Adjacency Array Weighted Adjacency Array Incidence Array Conclusions, Exercises, and References Manipulating Graphs with Matrices Introduction Matrix Indices and Values Composable Graph Operations and Linear Systems Matrix Graph Operations Overview Graph Algorithms and Diverse Semirings Conclusions, Exercises, and References Graph Analysis and Machine Learning Systems Introduction Data Representation Graph Construction Adjacency Array Graph Traversal Incidence Array Graph Traversal Vertex Degree Centrality Edge Degree Centrality Eigenvector Centrality Singular Value Decomposition PageRank Deep Neural Networks Conclusions, Exercises, and References Mathematical Foundations Visualizing the Algebra of Associative Arrays Associative Array Analogs of Matrix Operations Abstract Algebra for Computer Scientists and Engineers Depicting Mathematics Associative Array Class Diagrams Set Semiring Linear Algebra Ordered Sets Boolean Algebra Associative Array Algebra Conclusions, Exercises, and References Defining the Algebra of Associative Arrays Operations on Sets Ordered Sets Supremum and Infimum Lattice The Semirings of Interest Conclusions, Exercises, and References Structural Properties of Associative Arrays Estimating Structure Associative Array Formal Definition Padding Associative Arrays with Zeros Zero, Null, Zero-Sum-Free Properties of Matrices and Associative Arrays Properties of Zero Padding Support and Size Image and Rank Example: Music Example: Art Properties of Element-Wise Addition Properties of Element-Wise Multiplication Array Multiplication Closure of Operations between Arrays Conclusions, Exercises, and References Graph Construction and Graphical Patterns Introduction Adjacency and Incidence Array Definitions Adjacency Array Construction Graph Construction with Different Semirings Special Arrays and Graphs Key Ordering Algebraic Properties Subobject Properties Conclusions, Exercises, and References Linear Systems Survey of Common Transformations Array Transformations Identity Contraction Stretching Rotation Conclusions, Exercises, and References Maps and Bases Semimodules Linear Maps Linear Independence and Bases Existence of Bases Size of Bases Semialgebras and the Algebra of Arrays Conclusions, Exercises, and References Linearity of Associative Arrays The Null Space of Linear Maps Supremum-Blank Algebras Max-Blank Structure Theorem Examples of Supremum-Blank Algebras Explicit Computations of x(A,w) for Supremum-Blank Algebras Conclusions, Exercises, and References Eigenvalues and Eigenvectors Introduction Quasi-Inverses Existence of Eigenvalues for Idempotent Multiplication Strong Dependence and Characteristic Bipolynomial Eigenanalysis for Irreducible Matrices for Invertible Multiplication Eigen-Semimodules Singular Value Decomposition Conclusions, Exercises, and References Higher Dimensions d-Dimensional Associative Arrays Key Ordering and Two-Dimensional Projections Algebraic Properties Subarray Properties Conclusions, Exercises, and References Appendix: Notation Index Answers to Selected Exercises