تحلیل و جبر خطی: تجزیه مقدار منفرد و کاربردهای آن

This book provides an elementary analytically inclined journey to a fundamental result of linear algebra: the Singular Value Decomposition (SVD). SVD is a workhorse in many applications of linear algebra to data science. Four important applications relevant to data science are considered throughout the book: determining the subspace that “best” approximates a given set (dimension reduction of a data set); finding the “best” lower rank approximation of a given matrix (compression and general approximation problems); the Moore-Penrose pseudo-inverse (relevant to solving least squares problems); and the orthogonal Procrustes problem (finding the orthogonal transformation that most closely transforms a given collection to a given configuration), as well as its orientation-preserving version. The point of view throughout is analytic. Readers are assumed to have had a rigorous introduction to sequences and continuity. These are generalized and applied to linear algebraic ideas. Along the way to the SVD, several important results relevant to a wide variety of fields (including random matrices and spectral graph theory) are explored: the Spectral Theorem; minimax characterizations of eigenvalues; and eigenvalue inequalities. By combining analytic and linear algebraic ideas, readers see seemingly disparate areas interacting in beautiful and applicable ways. Cover Title page Copyright Contents Preface Pre-Requisites Notation Acknowledgements Chapter 1. Introduction 1.1. Why Does Everybody Say Linear Algebra is “Useful”? 1.2. Graphs and Matrices 1.3. Images 1.4. Data 1.5. Four “Useful” Applications Chapter 2. Linear Algebra and Normed Vector Spaces 2.1. Linear Algebra 2.2. Norms and Inner Products on a Vector Space 2.3. Topology on a Normed Vector Space 2.4. Continuity 2.5. Arbitrary Norms on R^{d} 2.6. Finite-Dimensional Normed Vector Spaces 2.7. Minimization: Coercivity and Continuity 2.8. Uniqueness of Minimizers: Convexity 2.9. Continuity of Linear Mappings Chapter 3. Main Tools 3.1. Orthogonal Sets 3.2. Projection onto (Closed) Subspaces 3.3. Separation of Convex Sets 3.4. Orthogonal Complements 3.5. The Riesz Representation Theorem and Adjoint Operators 3.6. Range and Null Spaces of L and L* 3.7. Four Problems, Revisited Chapter 4. The Spectral Theorem 4.1. The Spectral Theorem 4.2. Courant-Fischer-Weyl Min-Max Theorem for Eigenvalues 4.3. Weyl’s Inequalities for Eigenvalues 4.4. Eigenvalue Interlacing 4.5. Summary Chapter 5. The Singular Value Decomposition 5.1. The Singular Value Decomposition 5.2. Alternative Characterizations of Singular Values 5.3. Inequalities for Singular Values 5.4. Some Applications to the Topology of Matrices 5.5. Summary Chapter 6. Applications Revisited 6.1. The “Best” Subspace for Given Data 6.2. Least Squares and Moore-Penrose Pseudo-Inverse 6.3. Eckart-Young-Mirsky for the Operator Norm 6.4. Eckart-Young-Mirsky for the Frobenius Norm and Image Compression 6.5. The Orthogonal Procrustes Problem 6.6. Summary Chapter 7. A Glimpse Towards Infinite Dimensions Bibliography Index of Notation Index Back Cover I would like to first thank CWU's Department of Mathematics and all my colleagues there for making it possible for me to go on sabbatical, during which I wrote most of this book. Without that opportunity, this book would certainly not exist. Next, my thanks to all of the students who made it through my year long sequence in "applicable analysis":