PRODUCTIVE AND EFFICIENT DATA SCIENCE WITH PYTHON : with modularizing, memory profiles, and... parallel/gpu processing

This book focuses on the Python-based tools and techniques to help you become highly productive at all aspects of typical data science stacks such as statistical analysis, visualization, model selection, and feature engineering. You’ll review the inefficiencies and bottlenecks lurking in the daily business process and solve them with practical solutions. Automation of repetitive data science tasks is a key mindset that is promoted throughout the book. You’ll learn how to extend the existing coding practice to handle larger datasets with high efficiency with the help of advanced libraries and packages that already exist in the Python ecosystem. The book focuses on topics such as how to measure the memory footprint and execution speed of machine learning models, quality test a data science pipelines, and modularizing a data science pipeline for app development. You’ll review Python libraries which come in very handy for automating and speeding up the day-to-day tasks. In the end, you’ll understand and perform data science and machine learning tasks beyond the traditional methods and utilize the full spectrum of the Python data science ecosystem to increase productivity. What You’ll Learn Write fast and efficient code for data science and machine learning Build robust and expressive data science pipelines Measure memory and CPU profile for machine learning methods Utilize the full potential of GPU for data science tasks Handle large and complex data sets efficiently Who This Book Is For Data scientists, data analysts, machine learning engineers, Artificial intelligence practitioners, statisticians who want to take full advantage of Python ecosystem. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: What Is Productive and Efficient Data Science? A Typical Data Science Pipeline Typical Examples of Inefficient Practices in Data Science Iterating Over a pandas DataFrame Brute-Force for Loop Better Approaches: df.iterrows and df.values Scatterplot Everything in a Large Dataset Combinatorial Explosion Writing Similar Plotting Code Multiple Times Write a Generic Function Instead Not Writing A Test Module Some Pitfalls to Avoid Don’t Live in Ignorance. Measure Efficiency. Don’t Leave Your Code as Orphans. Modularize Them. The Python-Powered Data Science Legacy May Have a Problem Embrace OOP Principles As Much As You Can Don’t Be Limited by Hardware or Traditional Tools Local Hardware Memory Limitation Is a Real Issue GPU-Accelerated Computing Has Not Focused on Data Science as a Whole Always Explore Alternative Libraries/Frameworks Efficiency and Productivity Go Hand in Hand Measuring Efficiency Goes a Long Way Testing Reduces the Chance of Rework Planning ML Model Development Knowledge of GUI Programming/Web App Development Is Quite Helpful Skills and Attitude for Practicing Productive Data Science Summary Chapter 2: Better Programming Principles for Efficient Data Science The Concept of Time and Space Complexities plus Big-O Notation A Simple Example: Searching for an Element The Big-O Notation Complexities: Linear, Logarithmic, Quadratic, and More How Much Faster? What’s Beyond Linear? Why Complexity Matters for Data Science Image Data: Cubic-Complexity Algorithms Best Regression Model: Exponential Complexity Relative Growth Comparison AI Is Intractable, but It Works Inefficient Programming in Data Science Canonical Examples Use a Filter Instead of a for Loop Use Sets to Find Unique Elements Use a Specialized Data Structure for Counting Use the itertools Library for Combinatorial Structures Lessons Learned from the Examples Measuring Code Execution Timing Python’s time Module Is Your Friend Basic Usage Example Many Loops Needed for a Fast Code Block A Timing Decorator Using the Decorator to Measure Complexity Jupyter/IPython Magic Command %timeit: Execution Time for Single-Line Code %%timeit: Measuring Execution Time for a Block of Code in a Cell Summary Chapter 3: How to Use Python Data Science Packages More Productively Why NumPy Is Faster Than Regular Python Code and By How Much NumPy Arrays are Different NumPy Array vs. Native Python Computation NumPy and Native Python Implementation Conversion Adds Overhead Using NumPy Efficiently Conversion First, Operation Later Vectorize Logical Operations Use the Built-In Vectorize Function Avoid Using the .append Method Utilizing NumPy Reading Utilities Reading from a Flat Text File Utility for Tabular Data in a Text File Using pandas Productively Setting Values in a New DataFrame The .at or .iloc Methods Are Slow Use .values to Speed Things Up Significantly Specify Data Types Whenever Possible Iterating Over a DataFrame Brute-Force For Loop Better Approaches: df.iterrows and df.values Using Modern, Optimized File Formats Impressive Speed Improvement Read Only What Is Needed PyArrow to pandas and Back Other Miscellaneous Ideas Remove Orphan DataFrames Regularly Chaining Methods Using Specialized Libraries to Enhance Performance Efficient EDA with Matplotlib and Seaborn Embrace the Object-Oriented Nature of Matplotlib Two Approaches for Creating Panels with Subplots A Better Approach with a Clever Function Set and Control Image Quality Setting DPI Directly in plt.figure() Setting DPI and Output Format for Saving Figures Adjust Global Parameters Tricks with Seaborn Use Sampled Data for Large Datasets Use pandas Correlation with Seaborn heatmap Use Special Seaborn Methods to Reduce Work Summary Chapter 4: Writing Machine Learning Code More Productively Why (and How) to Modularize Code for Machine Learning Questions to Ask Yourself Start Simple with a Standard Data Science Flow A Scikit-learn Task Flow Example The Monolithic Example Little Boxes, Little Boxes... How to Use the Modular Code Systematic Evaluation of ML Algorithms in an Automated Fashion List of Classifiers Function to Automate Model Fitting How Does Automation Help? Decision Boundary Visualization The Custom Function Example Results Parametric Experimentation Other Scikit-learn Utilities and Techniques Hyperparameter Search Utilities Parallel Job Runner Out-of-the-box Visualization Methods Synthetic Data Generators Summary Chapter 5: Modular and Productive Deep Learning Code Modular Code and Object-Oriented Style for Productive DL Example of a Productive DL Task Flow Wrappers, Builders, Callbacks Modular Code for Fast Experimentation Business/Data Science Question Inherit from the Keras Callback Model Builder and Compile/Train Functions Visualization Function Final Analytics Code, Compact and Simple Turn the Scripts into a Utility Module Summary of Good Practices Streamline Image Classification Task Flow The Dataset Building the Data Generator Object Building the Convolutional Neural Net Model Training with the fit_generator Method Encapsulate All of This in a Single Function Testing the Utility Function Does It Work (Readily) for Another Dataset? Other Extensions Activation Maps in a Few Lines of Code Activation Maps Activation Maps with a Few Lines of Code Training Activation Another Library for Web-Based UI How Is This Productive Data Science? Hyperparameter Search with Scikit-learn Scikit-learn Enmeshes with Keras Data and (Preliminary) Keras Model The KerasClassifier Class Cross-Validation with the Scikit-learn API Grid Search with a Updated Model Summary Chapter 6: Build Your Own ML Estimator/Package Why Develop Your Own ML Package? A Data Scientist’s Example An Arithmetic Example Data Scientists Use OOP All the Time How Was It Made? Linear Regression Estimator—with a Twist How Do You Start Building This? Base Class Definition Adding Useful Methods The Fitting Method Testing the Method Prediction Method Testing Prediction Adding Utility Methods Method for Plotting True vs. Predicted Values All Kinds of Error Metrics Do More in the OOP Style Separate Plotting Classes More Supporting Classes and Syntactic Sugar Modularization: Importing the Class as a Module Publishing It as a Python Package Special Instructions for PyPI Hosting GitHub Integration Summary Chapter 7: Some Cool Utility Packages Build Pipelines Using pdpipe The Dataset Start Laying Pipes Chain Stages of Pipeline Simply by Adding Dropping Rows Based on Their Values scikit-learn and NLTK Stages Scaling Data with a scikit-learn Method Tokenizer from NLTK All Together Speeding Up NumPy and pandas What Is This Library? Speeding It Up Arithmetic Involving Two Arrays A Somewhat More Complex Operation Logical Expressions/Boolean Filtering Complex Numbers Impact of the Array Size The pandas eval Method How It Works, Supported Operators Discover Best-Fitting Distributions Quickly Simple Fitting Example Plot and Summary Be Careful with Small Datasets Other Things You Can Do Summary Chapter 8: Memory and Timing Profile Why Profile Memory Usage? A Common Scenario It’s Not the Model Size (or Compression) Scalene: A Neat Little Memory Profiler Basic Usage Features A Concrete Machine Learning Example Linear Regression Model What Happens as the Model and Data Scale? Deep Learning Model Key Approaches and Advice Key Advice Other Things You Can Do with Scalene Final Validation Is Sometimes Necessary Timing Profile with cProfile Basic Usage With a Function as an Argument Using the Profiler Class Data Science Workflow Profiling Summary Chapter 9: Scalable Data Science Common Problems for Scalability Out-of-Core (a.k.a. Out of Memory) Python Single Threading What Options Are Out There? Cloud Instances Google Colab pandas-Specific Tricks Load Only the Columns You Need Column-Specific Functions (If Applicable) Explicitly Specify/Convert Data Types Libraries for Parallel Processing Libraries for Handling Out-of-Core Datasets A Note About the Preferred OS Hands-On Example with Vaex Features at a Glance Basic Usage Example No Unnecessary Memory Copying Expressions and Virtual Columns Computation on a Multidimensional Grid Dynamic Visualizations Using Widgets and Other Plotting Libraries Vaex Preferred HDF5 Format Hands-On Examples with Modin Single CPU Core to Multi-Core Out-of-Core Processing Other Features of Modin Summary Chapter 10: Parallelized Data Science Parallel Computing for Data Science Single Core to Multi-Core CPUs What Is Parallel in Data Science? Parallel Data Science with Dask How Dask Works Under the Hood Dask Array Dask DataFrame Dask Bag Dask Task Graph Works on Many Types of Clusters Basic Usage Examples Array DataFrames Dask Bags Dask Distributed Client Dask Machine Learning Module What Problems Does It Address? Tight Integration with scikit-learn Parallel Computing with Ray Features and Ecosystem of Ray Simple Parallelization Example Ray Dataset for Distributed Loading and Compute Summary Chapter 11: GPU-Based Data Science for High Productivity The RAPIDS Ecosystem CuPy CuDF CuML CuGraph Hardware Story Choice of Environment and Setup CuPy vs. NumPy Looks and Works Just Like NumPy Much Faster Than NumPy Data (Array) Size Matters CuDF vs. pandas Data Reading from an URL Indexing, Filtering, and Grouping NumPy Array Conversion Simple Benchmarking of Speed Dask Integration, User-Defined Functions, and Other Features CuML vs. scikit-learn Classification with Random Forest K-Means Clustering Summary Chapter 12: Other Useful Skills to Master Understanding the Basics of Web Technologies A Consumer-Facing Layer All Useful Data Science Is Delivered Through Web Apps What Are Some Pathways to Learn? Building Simple Web Apps for Data Science Hands-On Example with Flask Hands-On Example with PyWebIO Other Options and GUI-Building Tools Going from Local to the Cloud Many Types of Cloud Services for Data Science Platform-as-a-Service Data-as-a-Service Bringing Cloud Power to a Local Environment Low-Code Libraries for Productive Data Science What Are These Low-Code Libraries? Example with PyCaret Summary Chapter 13: Wrapping It Up Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 What Was Not Discussed in This Book MLOps and DataOps Container Technologies Database Technologies General Advice for Upcoming Data Scientists Ask Questions and Learn Constantly If You Are a Beginner At a More Advanced Phase Learn a Diverse Set of Skills Read About Broad Topics at Every Chance Distinguish Yourself at a Job Interview Some Useful Resources A Data Scientist’s Amazing, Curated List of Useful Tricks and Tools Build Installable Software Packages Using Only Jupyter Notebooks Learn How to Integrate Unit Testing Principles Write Whole Programming and Technology Books Right from Your Jupyter Notebook Get Started with MLOps Understand the Multi-Faceted Complexity of a Real-Life Analytics Problem Begin a New Journey Index