توسعه پیشرفته پایتون: استفاده از ویژگی‌های قدرتمند زبان در برنامه‌های دنیای واقعی

This book builds on basic Python tutorials to explain various Python language features that aren’t routinely covered: from reusable console scripts that play double duty as micro-services by leveraging entry points, to using asyncio efficiently to collate data from a large number of sources. Along the way, it covers type-hint based linting, low-overhead testing and other automated quality checking to demonstrate a robust real-world development process.Some powerful aspects of Python are often documented with contrived examples that explain the feature as a standalone example only. By following the design and build of a real-world application example from prototype to production quality you'll see not only how the various pieces of functionality work but how they integrate as part of the larger system design process. In addition, you'll benefit from the kind of useful asides and library recommendations that are a staple of conference Q&A sessions at Python conferences as well as discussions of modern Python best practice and techniques to better produce clear code that is easily maintainable.__Advanced Python Development__ is intended for developers who can already write simple programs in Python and want to understand when it’s appropriate to use new and advanced language features and to do so in a confident manner. It is especially of use to developers looking to progress to a more senior level and to very experienced developers who have thus far used older versions of Python. **What You'll Learn*** Understand asynchronous programming * Examine developing plugin architectures * Work with type annotations * Review testing techniques * Explore packaging and dependency management **Who This Book Is For**Developers at the mid to senior level who already have Python experience. Table of Contents About the Author About the Technical Reviewers Acknowledgments Introduction Chapter 1: Prototyping and environments Prototyping in Python Prototyping with the REPL Prototyping with a Python script Prototyping with scripts and pdb Post-mortem debugging The breakpoint function Prototyping with Jupyter Notebooks Prototyping in this chapter Environment setup Setting up a new project Prototyping our scripts Installing dependencies Exporting to a .py file Building a command-line interface The sys module and argv argparse click Pushing the boundaries Remote kernels Developing code that cannot be run locally The completed script Summary Additional resources Chapter 2: Testing, checking, linting Testing When to write tests Creating formatting functions for improved testability pytest Unit, integration, and functional testing Pytest fixtures Categorizing test functions Coverage Type checking Installing mypy Adding type hints Subclasses and inheritance Generic types Debugging and overuse of typing When to use typing and when to avoid it Keeping type hints separate from code Linting Installing flake8 and black Fixing existing code Running automatically Running on pull requests Summary Additional resources Chapter 3: Packaging scripts Terminology Directory structure Setup scripts and metadata Dependencies Declarative configurations Things to avoid in setup.py Conditional dependencies Readme in metadata Version numbers Using setup.cfg Custom index servers Setting up pypiserver Durability Confidentiality Integrity Wheel formats and executing code on installation Creating wheels from existing distributions Installing the console script using entrypoints README, DEVELOP, and CHANGES Markdown format reStructured text format README CHANGES.md and versioning Semantic versioning Calendar versioning Upstream dependency version pins Loose pins Strict pins Which pinning scheme to use Uploading a version Configuring twine Summary Additional resources Chapter 4: From script to framework Writing a sensor plugin Developing the plugin Adding a new command option Subcommands Command options Error handling Off-loading parsing to Click with argument types Custom click argument types Canned options Allowing third-party sensor plugins Plugin detection using fixed names Plugin detection using entrypoints Configuration files Environment variables Approach for apd.sensors vs. similar programs Summary Additional resources Chapter 5: Alternative interfaces Web microservices WSGI API design Authentication Flask Python decorators Closures Modifying variables in parent scopes Basic decorators Decorators with arguments Decorator-based security Testing the view function Deployment Extending software as a third party Agreeing on an ad hoc signature with peers Abstract base classes Fallback strategies Adapter pattern Dynamic class generation Other serialization formats Bringing it all together Fixing the serialization problem in our code Tidying up Versioning APIs Testability Summary Additional resources Chapter 6: Aggregation process Cookiecutter Creating a new template Creating the aggregation package Database types Our example Object-relational mappers Versioning the database Other useful alembic commands Loading data New technologies Databases Custom attribute behavior Generators Summary Additional resources Chapter 7: Parallelization and async Nonblocking IO Making our code nonblocking Multithreading and multiprocessing Low-level threads Bytecode The GIL Locks and deadlocks Deadlocks Avoiding global state Collating data Passing data Other synchronization primitives Reentrant locks Conditions Barriers Event Semaphore ProcessPoolExecutors Making our code multithreaded AsyncIO async def await async for async with Async locking primitives Working with synchronous libraries Making our code asynchronous Comparison Making a choice Summary Additional resources Chapter 8: Advanced asyncio Testing async code Testing our code Test servers and pytest fixtures with teardown Fixture scoping Mocking objects for easier unit testing Mocks with branching logic Data classes contextlib Test methods Asynchronous databases Classic SQLAlchemy style Uncompiled mssql mysql Postgresql sqlite Using run_in_executor Querying data Avoiding complex queries Querying against views Alternatives Global variables in asynchronous code Summary Additional resources Chapter 9: Viewing the data Query functions Filtering data Multilevel iterators Additional filters Testing our query functions Parameterized tests Displaying multiple sensors Processing data Interactivity with Jupyter widgets Multiply nested synchronous and asynchronous code Tidying up Persistent endpoints Charting maps and geographic data New plot types Supporting map type charts in apd.aggregation Backward compatibility in data classes Drawing a custom map using the new configs Summary Additional resources Chapter 10: Speeding things up Optimizing a function Profiling and threads Interpreting the profile report Other profilers timeit line_profiler yappi Tracemalloc New Relic Optimizing control flow Visualizing profiling data Caching Cached properties Summary Additional resources Chapter 11: Fault tolerance Error handling Getting items from a container Abstract base classes Exception types TypeError and ValueError RuntimeError and SystemExit AssertionError Custom exceptions Creating new exception types Additional metadata Tracebacks involving multiple exceptions Exception in except or finally block raise from Testing for exception handling New behaviors Advanced mocking with unittest.Mock Warnings Warning filters Logging Nested loggers Custom actions Extra metadata Logging adapter LogRecord factory Logging filters Logging configuration Other handlers Audit logs Designing around problems Scheduling sensor lookups APIs and filtering Summary Additional resources Chapter 12: Callbacks and data analysis Generator data flow Generators that consume their own output Enhanced generators Using classes Using an enhanced generator to wrap an iterable Refactoring functions with excessive return values Queues Choosing a control flow Structure for our actions Analysis coroutines Ingesting data Running the analysis process Process status Callbacks Extending the actions available Summary Additional resources Epilogue Index This book builds on basic Python tutorials to explain various Python language features that aren’t routinely covered: from reusable console scripts that play double duty as micro-services by leveraging entry points, to using asyncio efficiently to collate data from a large number of sources. Along the way, it covers type-hint based linting, low-overhead testing and other automated quality checking to demonstrate a robust real-world development process. Some powerful aspects of Python are often documented with contrived examples that explain the feature as a standalone example only. By following the design and build of a real-world application example from prototype to production quality you'll see not only how the various pieces of functionality work but how they integrate as part of the larger system design process. In addition, you'll benefit from the kind of useful asides and library recommendations that are a staple of conference Q&A sessions at Python conferences as well as discussions of modern Python best practice and techniques to better produce clear code that is easily maintainable. Advanced Python Development is intended for developers who can already write simple programs in Python and want to understand when it’s appropriate to use new and advanced language features and to do so in a confident manner. It is especially of use to developers looking to progress to a more senior level and to very experienced developers who have thus far used older versions of Python. What You'll Learn Understand asynchronous programming Examine developing plugin architectures Work with type annotations Review testing techniques Explore packaging and dependency management Who This Book Is For Developers at the mid to senior level who already have Python experience. This book builds on basic Python tutorials to explain various Python language features that arent routinely covered: from reusable console scripts that play double duty as micro-services by leveraging entry points, to using asyncio efficiently to collate data from a large number of sources. Along the way, it covers type-hint based linting, low-overhead testing and other automated quality checking to demonstrate a robust real-world development process. Some powerful aspects of Python are often documented with contrived examples that explain the feature as a standalone example only. By following the design and build of a real-world application example from prototype to production quality you'll see not only how the various pieces of functionality work but how they integrate as part of the larger system design process. In addition, you'll benefit from the kind of useful asides and library recommendations that are a staple of conference Q & A sessions at Python conferences as well as discussions of modern Python best practice and techniques to better produce clear code that is easily maintainable. Advanced Python Development is intended for developers who can already write simple programs in Python and want to understand when its appropriate to use new and advanced language features and to do so in a confident manner. It is especially of use to developers looking to progress to a more senior level and to very experienced developers who have thus far used older versions of Python. You will: Understand asynchronous programming Examine developing plugin architectures Work with type annotations Review testing techniques Explore packaging and dependency management