Data Literacy With Python

The purpose of this book is to usher readers into the world of data, ensuring a comprehensive understanding of its nuances, intricacies, and complexities. With Python 3 as the primary medium, the book underscores the pivotal role of data in modernindustries, and how its adept management can lead to insightful decision-making. The book provides a quick introduction to foundational data-related tasks, priming the readers for more advanced concepts of model training introduced later on. Through detailed, step-by-step Python code examples, the reader will master training models, beginning with the kNN algorithm, and then smoothly transitioning to other classifiers, by tweaking mere lines of code. Tools like Sweetviz, Skimpy, Matplotlib, and Seaborn are introduced, offering readers a hands-on experience in rendering charts and graphs. Companion files with source code and data sets are available by writing to the publisher. At its heart, the book provides a swift introduction to foundational data-related tasks, priming the readers for more advanced concepts of model training introduced later on. Through detailed, step-by-step Python code examples, the readers will traverse the journey of training models, beginning with the kNN algorithm, and then smoothly transitioning to other classifiers, effortlessly, by tweaking mere lines of code. To derive the maximum value from this book, a foundational grasp of Python 3.x is requisite. While some sections might necessitate a preliminary understanding of the ‘awk’ utility, the majority of the content is dedicated to Python’s prowess. Familiarity with Pandas, especially its data frames, will further enhance the reader’s journey. Features: Introduces tools like Sweetviz, Skimpy, Matplotlib, and Seaborn offering readers a hands-on experience in rendering charts and graphs Companion files with numerous Python code samples Designed for individuals beginning their foray into Machine Learning, the language caters to a global audience. By intentionally steering clear of colloquialisms, and adopting a standard English approach, it ensures content clarity for readers, irrespective of their linguistic backgrounds. Front Cover Half-Title Page License, Disclaimer of Liability, and Limited Warranty Title Page Copyright Page Dedication Contents Preface Chapter 1: Working With Data What Is Data Literacy? Exploratory Data Analysis (EDA) Where Do We Find Data? Dealing With Data: What Can Go Wrong? Explanation of Data Types Working With Data Types What Is Drift? Discrete Data Versus Continuous Data “Binning” Data Values Correlation Working With Synthetic Data Summary References Chapter 2: Outlier and Anomaly Detection Working With Outliers Finding Outliers With NumPy Finding Outliers With Pandas Fraud Detection Techniques for Anomaly Detection Working With Imbalanced Datasets Summary Chapter 3: Cleaning Datasets Analyzing Missing Data Pandas, CSV Files, and Missing Data Missing Data and Imputation Data Normalization Handling Categorical Data Data Wrangling Summary Chapter 4: Introduction to Statistics Basic Concepts in Statistics Random Variables Multiple Random Variables Basic Concepts in Statistics The Variance and Standard Deviation Sampling Techniques for a Population The Confusion Matrix Calculating Expected Values Summary References Chapter 5: Matplotlib and Seaborn What Is Data Visualization? What Is Matplotlib? Matplotlib Styles Display Attribute Values Color Values in Matplotlib Cubed Numbers in Matplotlib Horizontal Lines in Matplotlib Slanted Lines in Matplotlib Parallel Slanted Lines in Matplotlib Lines and Labeled Vertices in Matplotlib A Dotted Grid in Matplotlib Lines in a Grid in Matplotlib Two Lines and a Legend in Matplotlib Loading Images in Matplotlib A Set of Line Segments in Matplotlib Plotting Multiple Lines in Matplotlib A Histogram in Matplotlib Plot Bar Charts Plot a Pie Chart Heat Maps Save Plot as a PNG File Working With SweetViz Working With Skimpy Working With Seaborn Seaborn Dataset Names Seaborn Built-In Datasets The Iris Dataset in Seaborn The Titanic Dataset in Seaborn Extracting Data From Titanic Dataset in Seaborn Visualizing a Pandas DataFrame in Seaborn Seaborn Heat Maps Seaborn Pair Plots Summary Appendix A: Introduction to Python Tools for Python Python Installation Setting the PATH Environment Variable (Windows Only) Launching Python on Your Machine Python Identifiers Lines, Indentation, and Multilines Quotation and Comments in Python Saving Your Code in a Module Some Standard Modules in Python The help() and dir() Functions Compile Time and Runtime Code Checking Simple Data Types in Python Working With Numbers Working With Fractions Unicode and UTF-8 Working With Unicode Working With Strings Uninitialized Variables and the Value None in Python Slicing and Splicing Strings Search and Replace a String in Other Strings Remove Leading and Trailing Characters Printing Text Without NewLine Characters Text Alignment Working With Dates Exception Handling in Python Handling User Input Python and Emojis (Optional) Command-Line Arguments Summary Appendix B: Introduction to Pandas What Is Pandas? A Pandas DataFrame With a NumPy Example Describing a Pandas DataFrame Pandas Boolean DataFrames Pandas DataFrames and Random Numbers Reading CSV Files in Pandas The loc() and iloc() Methods in Pandas Converting Categorical Data to Numeric Data Matching and Splitting Strings in Pandas Converting Strings to Dates in Pandas Working With Date Ranges in Pandas Detecting Missing Dates in Pandas Interpolating Missing Dates in Pandas Other Operations With Dates in Pandas Merging and Splitting Columns in Pandas Reading HTML Web Pages in Pandas Saving a Pandas DataFrame as an HTML Web Page Summary Index