Test-Driven Development with React

Master these effective techniques to reduce costs and run times, handle huge datasets, and implement complex machine learning applications efficiently in Python. Fast Python for Data Science is a hands-on guide to writing Python code that can process more data, faster, and with less resources. It takes a holistic approach to Python performance, showing you how your code, libraries, and computing architecture interact and can be optimized together. Written for experienced practitioners, Fast Python for Data Science dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in lower-level Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. Fast Python for Data Science MEAP V03 Copyright welcome brief contents Chapter 1: The need for efficient computing and data storage 1.1 The overwhelming need for efficient computing in Python 1.2 The impact of modern computing architectures on high performance computing 1.2.1 Changes inside the computer 1.2.2 Changes in the network 1.2.3 The cloud 1.3 Working with Python’s limitations 1.3.1 The Global Interpreter Lock (GIL) 1.4 What will you learn from this book 1.5 The reader for this book 1.6 Summary Chapter 2: Extracting maximum performance from built-in features 2.1 Introducing the project dataset 2.1.1 An architecture for big data processing 2.1.2 Preparing the data 2.2 Profiling code to detect performance bottlenecks 2.2.1 Using Python’s built-in profiling module 2.2.2 Visualizing profiling information 2.2.3 Line profiling 2.3 Optimizing basic data structures for speed: lists, sets, dictionaries 2.3.1 Performance of list searches 2.3.2 Using the bisect module 2.3.3 Content aware search approaches 2.3.4 Searching using sets or dictionaries 2.3.5 List complexity in Python 2.4 Finding excessive memory allocation 2.4.1 Navigating the minefield of Python memory estimation 2.4.2 Using more compact representations 2.4.3 Packing many observations in a number 2.4.4 Use the array module 2.4.5 Systematizing what we have learned: Estimating the memory usage of Python objects 2.5 Using laziness and generators for big-data pipelining 2.5.1 Using generators instead of standard functions 2.5.2 Enabling code pipelining with generators 2.6 Summary Chapter 3: Concurrency, parallelism and asynchronous processing in Python 3.1 Writing the scaffold of an asynchronous server 3.1.1 The implementation of the scaffold for communicating with clients 3.1.2 Programming with coroutines 3.1.3 Sending complex data from simple synchronous client 3.1.4 Alternative approaches 3.2 Implementing the first MapReduce engine 3.2.1 Understanding MapReduce frameworks 3.2.2 Developing a very simple test scenario 3.2.3 Implementing a too-simple MapReduce framework 3.3 Implementing a concurrent version of a MapReduce engine 3.3.1 Using concurrent.futures to implement a threaded server 3.3.2 Asynchronous execution with Futures 3.3.3 The GIL and multi-threading 3.4 Using multi-processing to implement MapReduce 3.4.1 A solution based on concurrent.futures 3.4.2 A solution based on the multiprocessing module 3.4.3 Monitoring the progress of the multiprocessing solution 3.4.4 Transferring data in chunks 3.5 Tying it all together: an asynchronous multi-threaded and multi-processing MapReduce server 3.5.1 Architecting a complete high-performance solution 3.5.2 Creating a robust version of the server 3.6 Summary Chapter 4: Using NumPy more efficiently 4.1 Understanding NumPy from a performance perspective 4.1.1 Copies and views 4.1.2 Understanding NumPy’s view machinery 4.1.3 Making use of views for efficiency 4.2 Using array programming 4.2.1 Broadcasting in NumPy 4.2.2 Applying array programming to image manipulation 4.2.3 Developing a "vectorized mentality" 4.3 Tuning NumPy’s internal architecture for performance 4.3.1 An overview of NumPy dependencies 4.3.2 How to tune NumPy in your Python distribution 4.3.3 Threads in NumPy 4.4 Summary Appendix: Setting up the environment A.1 Code style and organization Notes Master Python techniques and libraries to reduce run times, efficiently handle huge datasets, and optimize execution for complex machine learning applications. Fast Python is a toolbox of techniques for high performance Python including: Writing efficient pure-Python code Optimizing the NumPy and pandas libraries Rewriting critical code in Cython Designing persistent data structures Tailoring code for different architectures Implementing Python GPU computing Fast Python is your guide to optimizing every part of your Python-based data analysis process, from the pure Python code you write to managing the resources of modern hardware and GPUs. You'll learn to rewrite inefficient data structures, improve underperforming code with multithreading, and simplify your datasets without sacrificing accuracy. Written for experienced practitioners, this book dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working. About the Technology Face it. Slow code will kill a big data project. Fast pure-Python code, optimized libraries, and fully utilized multiprocessor hardware are the price of entry for machine learning and large-scale data analysis. What you need are reliable solutions that respond faster to computing requirements while using less resources, and saving money. About the Book Fast Python is a toolbox of techniques for speeding up Python, with an emphasis on big data applications. Following the clear examples and precisely articulated details, you'll learn how to use common libraries like NumPy and pandas in more performant ways and transform data for efficient storage and I/O. More importantly, Fast Python takes a holistic approach to performance, so you'll see how to optimize the whole system, from code to architecture. What's Inside Rewriting critical code in Cython Designing persistent data structures Tailoring code for different architectures Implementing Python GPU computing About the Reader For intermediate Python programmers familiar with the basics of concurrency. About the Author Tiago Antão is one of the co-authors of Biopython, a major bioinformatics package written in Python. Table of Contents: PART 1 - FOUNDATIONAL APPROACHES 1 An urgent need for efficiency in data processing 2 Extracting maximum performance from built-in features 3 Concurrency, parallelism, and asynchronous processing 4 High-performance NumPy PART 2 - HARDWARE 5 Re-implementing critical code with Cython 6 Memory hierarchy, storage, and networking PART 3 - APPLICATIONS AND LIBRARIES FOR MODERN DATA PROCESSING 7 High-performance pandas and Apache Arrow 8 Storing big data PART 4 - ADVANCED TOPICS 9 Data analysis using GPU computing 10 Analyzing big data with Dask Master Python techniques and libraries to reduce run times, efficiently handle huge datasets, and optimize execution for complex machine learning applications. Fast Python is a toolbox of techniques for high performance Python Fast Python is your guide to optimizing every part of your Python-based data analysis process, from the pure Python code you write to managing the resources of modern hardware and GPUs. You'll learn to rewrite inefficient data structures, improve underperforming code with multithreading, and simplify your datasets without sacrificing accuracy. Written for experienced practitioners, this book dives right into practical solutions for improving computation and storage efficiency. You'll experiment with fun and interesting examples such as rewriting games in Cython and implementing a MapReduce framework from scratch. Finally, you'll go deep into Python GPU computing and learn how modern hardware has rehabilitated some former antipatterns and made counterintuitive ideas the most efficient way of working. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology Face it. Slow code will kill a big data project. Fast pure-Python code, optimized libraries, and fully utilized multiprocessor hardware are the price of entry for machine learning and large-scale data analysis. What you need are reliable solutions that respond faster to computing requirements while using less resources, and saving money. About the Book Fast Python is a toolbox of techniques for speeding up Python, with an emphasis on big data applications. Following the clear examples and precisely articulated details, youll learn how to use common libraries like NumPy and pandas in more performant ways and transform data for efficient storage and I/O. More importantly, Fast Python takes a holistic approach to performance, so youll see how to optimize the whole system, from code to architecture . Whats Inside About the Reader For intermediate Python programmers familiar with the basics of concurrency. About the Author Tiago Anto is one of the co-authors of Biopython, a major bioinformatics package written in Python. Table of PART 1 - FOUNDATIONAL APPROACHES 1 An urgent need for efficiency in data processing 2 Extracting maximum performance from built-in features 3 Concurrency, parallelism, and asynchronous processing 4 High-performance NumPy PART 2 - HARDWARE 5 Re-implementing critical code with Cython 6 Memory hierarchy, storage, and networking PART 3 - APPLICATIONS AND LIBRARIES FOR MODERN DATA PROCESSING 7 High-performance pandas and Apache Arrow 8 Storing big data PART 4 - ADVANCED TOPICS 9 Data analysis using GPU computing 10 Analyzing big data with Dask **Master these effective techniques to reduce costs and run times, handle huge datasets, and implement complex machine learning applications efficiently in Python.** __Fast Python for Data Science____Fast Python for Data Science__