Expert Python Programming: Master Python by learning the best coding practices and advanced programming concepts, 4th Edition. Code

Attain a deep understanding of building, maintaining, packaging, and shipping robust Python applications Key Features • Discover the new features of Python, such as dictionary merge, the zoneinfo module, and structural pattern matching • Create manageable code to run in various environments with different sets of dependencies • Implement effective Python data structures and algorithms to write, test, and optimize code Book Description Python is used in a wide range of domains owing to its simple yet powerful nature. Although writing Python code is easy, making it readable, reusable, and easy to maintain can be challenging. Complete with best practices, useful tools, and standards implemented by professional Python developers, this fourth edition will help you in not only overcoming such challenges but also learning Python’s latest features and advanced concepts. The book begins with a warm-up, where you will catch-up with the latest Python improvements, syntax elements, and interesting tools to boost your development efficiency. Further, the initial few chapters should allow experienced programmers coming from different languages to safely land in the Python ecosystem. As you progress, you will explore common software design patterns and various programming methodologies, such as event-driven programming, concurrency, and metaprogramming. You will also go through complex code examples and try to solve meaningful problems by bridging Python with C and C++, writing extensions that benefit from the strengths of multiple languages. Finally, you will understand the complete lifetime of any application after it goes live. By the end of the book, you should be proficient in writing efficient and maintainable Python code. What you will learn • Explore modern ways of setting up repeatable and consistent Python development environments • Effectively package Python code for community and production use • Learn modern syntax elements of Python programming, such as f-strings, enums, and lambda functions • Demystify metaprogramming in Python with metaclasses • Write concurrent code in Python • Extend and integrate Python with code written in different languages Who This Book Is For The Python programming book is intended for expert programmers who want to learn Python’s advanced-level concepts and latest features. Anyone who has basic Python skills should be able to follow the content of the book, although it might require some additional effort from less experienced programmers. It should also be a good introduction to Python 3.9 for those who are still a bit behind and continue to use other older versions. Cover Copyright Contributors Table of Contents Preface Chapter 1: Current Status of Python Where are we now and where are we going? What to do with Python 2 Keeping up to date PEP documents Active communities Other resources Summary Chapter 2: Modern Python Development Environments Technical requirements Python's packaging ecosystem Installing Python packages using pip Isolating the runtime environment Application-level isolation versus system-level isolation Application-level environment isolation Poetry as a dependency management system System-level environment isolation Containerization versus virtualization Virtual environments using Docker Writing your first Dockerfile Running containers Setting up complex environments Useful Docker and Docker Compose recipes for Python Virtual development environments using Vagrant Popular productivity tools Custom Python shells Using IPython Incorporating shells in your own scripts and programs Interactive debuggers Other productivity tools Summary Chapter 3: New Things in Python Technical requirements Recent language additions Dictionary merge and update operators Alternative – Dictionary unpacking Alternative – ChainMap from the collections module Assignment expressions Type-hinting generics Positional-only parameters zoneinfo module graphlib module Not that new, but still shiny breakpoint() function Development mode Module-level __getattr__() and __dir__() functions Formatting strings with f-strings Underscores in numeric literals secrets module What may come in the future? Union types with the | operator Structural pattern matching Summary Chapter 4: Python in Comparison with Other Languages Technical requirements Class model and object-oriented programming Accessing super-classes Multiple inheritance and Method Resolution Order Class instance initialization Attribute access patterns Descriptors Real-life example – lazily evaluated attributes Properties Dynamic polymorphism Operator overloading Dunder methods (language protocols) Comparison to C++ Function and method overloading Single-dispatch functions Data classes Functional programming Lambda functions The map(), filter(), and reduce() functions Partial objects and partial functions Generators Generator expressions Decorators Enumerations Summary Chapter 5: Interfaces, Patterns, and Modularity Technical requirements Interfaces A bit of history: zope.interface Using function annotations and abstract base classes Using collections.abc Interfaces through type annotations Inversion of control and dependency injection Inversion of control in applications Using dependency injection frameworks Summary Chapter 6: Concurrency Technical requirements What is concurrency? Multithreading What is multithreading? How Python deals with threads When should we use multithreading? Application responsiveness Multiuser applications Work delegation and background processing An example of a multithreaded application Using one thread per item Using a thread pool Using two-way queues Dealing with errors in threads Throttling Multiprocessing The built-in multiprocessing module Using process pools Using multiprocessing.dummy as the multithreading interface Asynchronous programming Cooperative multitasking and asynchronous I/O Python async and await keywords A practical example of asynchronous programming Integrating non-asynchronous code with async using futures Executors and futures Using executors in an event loop Summary Chapter 7: Event-Driven Programming Technical requirements What exactly is event-driven programming? Event-driven != asynchronous Event-driven programming in GUIs Event-driven communication Various styles of event-driven programming Callback-based style Subject-based style Topic-based style Event-driven architectures Event and message queues Summary Chapter 8: Elements of Metaprogramming Technical requirements What is metaprogramming? Using decorators to modify function behavior before use One step deeper: class decorators Intercepting the class instance creation process Metaclasses The general syntax Metaclass usage Metaclass pitfalls Using the __init__subclass__() method as an alternative to metaclasses Code generation exec, eval, and compile The abstract syntax tree Import hooks Notable examples of code generation in Python Falcon's compiled router Hy Summary Chapter 9: Bridging Python with C and C++ Technical requirements C and C++ as the core of Python extensibility Compiling and loading Python C extensions The need to use extensions Improving performance in critical code sections Integrating existing code written in different languages Integrating third-party dynamic libraries Creating efficient custom datatypes Writing extensions Pure C extensions A closer look at the Python/C API Calling and binding conventions Exception handling Releasing GIL Reference counting Writing extensions with Cython Cython as a source-to-source compiler Cython as a language Downsides of using extensions Additional complexity Harder debugging Interfacing with dynamic libraries without extensions The ctypes module Loading libraries Calling C functions using ctypes Passing Python functions as C callbacks CFFI Summary Chapter 10: Testing and Quality Automation Technical requirements The principles of test-driven development Writing tests with pytest Test parameterization pytest's fixtures Using fakes Mocks and the unittest.mock module Quality automation Test coverage Style fixers and code linters Static type analysis Mutation testing Useful testing utilities Faking realistic data values Faking time values Summary Chapter 11: Packaging and Distributing Python Code Technical requirements Packaging and distributing libraries The anatomy of a Python package setup.py setup.cfg MANIFEST.in Essential package metadata Trove classifiers Types of package distributions sdist distributions bdist and wheel distributions Registering and publishing packages Package versioning and dependency management The SemVer standard for semantic versioning CalVer for calendar versioning Installing your own packages Installing packages directly from sources Installing packages in editable mode Namespace packages Package scripts and entry points Packaging applications and services for the web The Twelve-Factor App manifesto Leveraging Docker Handling environment variables The role of environment variables in application frameworks Creating standalone executables When standalone executables are useful Popular tools PyInstaller cx_Freeze py2exe and py2app Security of Python code in executable packages Summary Chapter 12: Observing Application Behavior and Performance Technical requirements Capturing errors and logs Python logging essentials Logging system components Logging configuration Good logging practices Distributed logging Capturing errors for later review Instrumenting code with custom metrics Using Prometheus Distributed application tracing Distributed tracing with Jaeger Summary Chapter 13: Code Optimization Technical requirements Common culprits for bad performance Code complexity Cyclomatic complexity The big O notation Excessive resource allocation and leaks Excessive I/O and blocking operations Code profiling Profiling CPU usage Macro-profiling Micro-profiling Profiling memory usage Using the objgraph module C code memory leaks Reducing complexity by choosing appropriate data structures Searching in a list Using sets Using the collections module deque defaultdict namedtuple Leveraging architectural trade-offs Using heuristics and approximation algorithms Using task queues and delayed processing Using probabilistic data structures Caching Deterministic caching Non-deterministic caching Summary Why subscribe? Packt Page Other Books You May Enjoy Index Gain a deep understanding of building, maintaining, packaging, and shipping robust Python applicationsKey FeaturesDiscover the new features of Python, such as dictionary merge, the zoneinfo module, and structural pattern matchingCreate manageable code to run in various environments with different sets of dependenciesImplement effective Python data structures and algorithms to write, test, and optimize codeBook DescriptionThis new edition of Expert Python Programming provides you with a thorough understanding of the process of building and maintaining Python apps. Complete with best practices, useful tools, and standards implemented by professional Python developers, this fourth edition has been extensively updated. Throughout this book, you'll get acquainted with the latest Python improvements, syntax elements, and interesting tools to boost your development efficiency.The initial few chapters will allow experienced programmers coming from different languages to transition to the Python ecosystem. You will explore common software design patterns and various programming methodologies, such as event-driven programming, concurrency, and metaprogramming. You will also go through complex code examples and try to solve meaningful problems by bridging Python with C and C++, writing extensions that benefit from the strengths of multiple languages. Finally, you will understand the complete lifetime of any application after it goes live, including packaging and testing automation.By the end of this book, you will have gained actionable Python programming insights that will help you effectively solve challenging problems.What you will learnExplore modern ways of setting up repeatable and consistent Python development environmentsEffectively package Python code for community and production useLearn modern syntax elements of Python programming, such as f-strings, enums, and lambda functionsDemystify metaprogramming in Python with metaclassesWrite concurrent code in PythonExtend and integrate Python with code written in C and C++Who this book is forThe Python programming book is intended for expert programmers who want to learn Python's advanced-level concepts and latest features.Anyone who has basic Python skills should be able to follow the content of the book, although it might require some additional effort from less experienced programmers. It should also be a good introduction to Python 3.9 for those who are still a bit behind and continue to use other older versions. Attain a deep understanding of building, maintaining, packaging, and shipping robust Python applications Key Features Discover the new features of Python, such as dictionary merge, the zoneinfo module, and structural pattern matching Create manageable code to run in various environments with different sets of dependencies Implement effective Python data structures and algorithms to write, test, and optimize code Book Description Python is used in a wide range of domains owing to its simple yet powerful nature. Although writing Python code is easy, making it readable, reusable, and easy to maintain can be challenging. Complete with best practices, useful tools, and standards implemented by professional Python developers, this fourth edition will help you in not only overcoming such challenges but also learning Python's latest features and advanced concepts. The book begins with a warm-up, where you will catch-up with the latest Python improvements, syntax elements, and interesting tools to boost your development efficiency. Further, the initial few chapters should allow experienced programmers coming from different languages to safely land in the Python ecosystem. As you progress, you will explore common software design patterns and various programming methodologies, such as event-driven programming, concurrency, and metaprogramming. You will also go through complex code examples and try to solve meaningful problems by bridging Python with C and C++, writing extensions that benefit from the strengths of multiple languages. Finally, you will understand the complete lifetime of any application after it goes live. By the end of the book, you should be proficient in writing efficient and maintainable Python code. What you will learn Explore modern ways of setting up repeatable and consistent Python development environments Effectively package Python code for community and production use Learn about modern syntax elements of Python programming, such as f-strings, dataclasses, enums, and lambda functions Demystify metaprogramming in Python with metaclasses Write concurrent code in Python Monitor and optimize the performance of Python application Extend and integrate Python with code written in different languages Who this book is for The Python programming book is intended for expert programmers who want to learn Python's advanced-level concepts and latest features. Anyone who has basic Python skills should be able to follow the content of the book, ..