Clean Code in Python : Refactor Your Legacy Code Base

Getting the most out of Python to improve your codebase Key Features Save maintenance costs by learning to fix your legacy codebase Learn the principles and techniques of refactoring Apply microservices to your legacy systems by implementing practical techniques Book Description Python is currently used in many different areas such as software construction, systems administration, and data processing. In all of these areas, experienced professionals can find examples of inefficiency, problems, and other perils, as a result of bad code. After reading this book, readers will understand these problems, and more importantly, how to correct them. The book begins by describing the basic elements of writing clean code and how it plays an important role in Python programming. You will learn about writing efficient and readable code using the Python standard library and best practices for software design. You will learn to implement the SOLID principles in Python and use decorators to improve your code. The book delves more deeply into object oriented programming in Python and shows you how to use objects with descriptors and generators. It will also show you the design principles of software testing and how to resolve software problems by implementing design patterns in your code. In the final chapter we break down a monolithic application to a microservice one, starting from the code as the basis for a solid platform. By the end of the book, you will be proficient in applying industry approved coding practices to design clean, sustainable and readable Python code. What you will learn Set up tools to effectively work in a development environment Explore how the magic methods of Python can help us write better code Examine the traits of Python to create advanced object-oriented design Understand removal of duplicated code using decorators and descriptors Effectively refactor code with the help of unit tests Learn to implement the SOLID principles in Python Who this book is for This book will appeal to team leads, software architects and senior software engineers who would like to work on their legacy systems to save cost and improve efficiency. A strong understanding of Programming is assumed. Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and registe .. Cover 1 Title Page 2 Copyright and Credits 3 Dedication 4 Packt Upsell 5 Contributors 6 Table of Contents 8 Preface 14 Chapter 1: Introduction, Code Formatting, and Tools 20 The meaning of clean code 21 The importance of having clean code 21 The role of code formatting in clean code 22 Adhering to a coding style guide on your project 23 Docstrings and annotations 25 Docstrings 26 Annotations 29 Do annotations replace docstrings? 31 Configuring the tools for enforcing basic quality gates 33 Type hinting with Mypy 33 Checking the code with Pylint 34 Setup for automatic checks 34 Summary 37 Chapter 2: Pythonic Code 38 Indexes and slices 39 Creating your own sequences 41 Context managers 42 Implementing context managers 45 Properties, attributes, and different types of methods for objects 48 Underscores in Python 48 Properties 51 Iterable objects 53 Creating iterable objects 53 Creating sequences 56 Container objects 58 Dynamic attributes for objects 59 Callable objects 61 Summary of magic methods 62 Caveats in Python 62 Mutable default arguments 63 Extending built-in types 64 Summary 66 References 66 Chapter 3: General Traits of Good Code 68 Design by contract 69 Preconditions 71 Postconditions 72 Pythonic contracts 72 Design by contract – conclusions 72 Defensive programming 73 Error handling 74 Value substitution 74 Exception handling 75 Handle exceptions at the right level of abstraction 77 Do not expose tracebacks 79 Avoid empty except blocks 80 Include the original exception 81 Using assertions in Python 82 Separation of concerns 83 Cohesion and coupling 84 Acronyms to live by 85 DRY/OAOO 85 YAGNI 87 KIS 87 EAFP/LBYL 89 Composition and inheritance 90 When inheritance is a good decision 91 Anti-patterns for inheritance 92 Multiple inheritance in Python 95 Method Resolution Order (MRO) 95 Mixins 97 Arguments in functions and methods 98 How function arguments work in Python 98 How arguments are copied to functions 99 Variable number of arguments 100 The number of arguments in functions 104 Function arguments and coupling 104 Compact function signatures that take too many arguments 105 Final remarks on good practices for software design 106 Orthogonality in software 107 Structuring the code 108 Summary 109 References 110 Chapter 4: The SOLID Principles 112 Single responsibility principle 112 A class with too many responsibilities 113 Distributing responsibilities 115 The open/closed principle 116 Example of maintainability perils for not following the open/closed principle 116 Refactoring the events system for extensibility 118 Extending the events system 120 Final thoughts about the OCP 122 Liskov's substitution principle 123 Detecting LSP issues with tools 124 Detecting incorrect datatypes in method signatures with Mypy 124 Detecting incompatible signatures with Pylint 126 More subtle cases of LSP violations 126 Remarks on the LSP 129 Interface segregation 130 An interface that provides too much 131 The smaller the interface, the better 131 How small should an interface be? 132 Dependency inversion 132 A case of rigid dependencies 133 Inverting the dependencies 134 Summary 135 References 136 Chapter 5: Using Decorators to Improve Our Code 137 What are decorators in Python? 137 Decorate functions 139 Decorate classes 140 Other types of decorator 144 Passing arguments to decorators 144 Decorators with nested functions 145 Decorator objects 147 Good uses for decorators 148 Transforming parameters 148 Tracing code 149 Effective decorators – avoiding common mistakes 149 Preserving data about the original wrapped object 149 Dealing with side-effects in decorators 152 Incorrect handling of side-effects in a decorator 152 Requiring decorators with side-effects 154 Creating decorators that will always work 156 The DRY principle with decorators 159 Decorators and separation of concerns 160 Analyzing good decorators 162 Summary 163 References 164 Chapter 6: Getting More Out of Our Objects with Descriptors 165 A first look at descriptors 165 The machinery behind descriptors 166 Exploring each method of the descriptor protocol 169 __get__(self, instance, owner) 169 __set__(self, instance, value) 170 __delete__(self, instance) 172 __set_name__(self, owner, name) 174 Types of descriptors 176 Non-data descriptors 176 Data descriptors 178 Descriptors in action 181 An application of descriptors 181 A first attempt without using descriptors 181 The idiomatic implementation 182 Different forms of implementing descriptors 185 The issue of global shared state 185 Accessing the dictionary of the object 186 Using weak references 187 More considerations about descriptors 188 Reusing code 188 Avoiding class decorators 189 Analysis of descriptors 193 How Python uses descriptors internally 193 Functions and methods 193 Built-in decorators for methods 196 Slots 198 Implementing descriptors in decorators 198 Summary 199 References 200 Chapter 7: Using Generators 201 Technical requirements 201 Creating generators 202 A first look at generators 202 Generator expressions 205 Iterating idiomatically 206 Idioms for iteration 206 The next() function 208 Using a generator 209 Itertools 209 Simplifying code through iterators 210 Repeated iterations 211 Nested loops 211 The iterator pattern in Python 213 The interface for iteration 213 Sequence objects as iterables 214 Coroutines 216 The methods of the generator interface 217 close() 217 throw(ex_type[, ex_value[, ex_traceback]]) 218 send(value) 219 More advanced coroutines 222 Returning values in coroutines 222 Delegating into smaller coroutines – the yield from syntax 223 The simplest use of yield from 224 Capturing the value returned by a sub-generator 225 Sending and receiving data to and from a sub-generator 226 Asynchronous programming 228 Summary 230 References 231 Chapter 8: Unit Testing and Refactoring 232 Design principles and unit testing 232 A note about other forms of automated testing 234 Unit testing and agile software development 235 Unit testing and software design 235 Defining the boundaries of what to test 238 Frameworks and tools for testing 239 Frameworks and libraries for unit testing 239 unittest 241 Parametrized tests 243 pytest 245 Basic test cases with pytest 246 Parametrized tests 247 Fixtures 248 Code coverage 249 Setting up rest coverage 249 Caveats of test coverage 250 Mock objects 251 A fair warning about patching and mocks 252 Using mock objects 252 Types of mocks 253 A use case for test doubles 254 Refactoring 256 Evolving our code 257 Production code isn't the only thing that evolves 259 More about unit testing 260 Property-based testing 261 Mutation testing 261 A brief introduction to test-driven development 264 Summary 265 References 265 Chapter 9: Common Design Patterns 266 Considerations for design patterns in Python 267 Design patterns in action 268 Creational patterns 269 Factories 269 Singleton and shared state (monostate) 270 Shared state 270 The borg pattern 273 Builder 275 Structural patterns 275 Adapter 276 Composite 277 Decorator 279 Facade 281 Behavioral patterns 282 Chain of responsibility 283 The template method 285 Command 286 State 287 The null object pattern 293 Final thoughts about design patterns 295 The influence of patterns over the design 295 Names in our models 296 Summary 297 References 298 Chapter 10 : Clean Architecture 299 From clean code to clean architecture 299 Separation of concerns 300 Abstractions 301 Software components 303 Packages 303 Containers 306 Use case 308 The code 309 Domain models 309 Calling from the application 311 Adapters 313 The services 313 Analysis 317 The dependency flow 317 Limitations 318 Testability 318 Intention revealing 319 Summary 319 References 320 Summing it all up 320 Other Books You May Enjoy 321 Index 324 How to build useful, real-world applications in the Python programming language About This Book Deliver scalable and high-performing applications in Python. Delve into the great ecosystem of Python frameworks and libraries through projects that you will build with this book. This comprehensive guide will help you demonstrate the power of Python by building practical projects. Who This Book Is For This book is for software developers who are familiar with Python and want to gain hands-on experience with web and software development projects. A basic knowledge of Python programming is required. What You Will Learn Learn object-oriented and functional programming concepts while developing projects The dos and don'ts of storing passwords in a database Develop a fully functional website using the popular Django framework Use the Beautiful Soup library to perform web scrapping Get started with cloud computing by building microservice and serverless applications in AWS Develop scalable and cohesive microservices using the Nameko framework Create service dependencies for Redis and PostgreSQL In Detail Python is a very powerful, high-level, object-oriented programming language. It's known for its simplicity and huge community support. Python Programming Blueprints will help you build useful, real-world applications using Python. In this book, we will cover some of the most common tasks that Python developers face on a daily basis, including performance optimization and making web applications more secure. We will familiarize ourselves with the associated software stack and master asynchronous features in Python. We will build a weather application using command-line parsing. We will then move on to create a Spotify remote control where we'll use OAuth and the Spotify Web API. The next project will cover reactive extensions by teaching you how to cast votes on Twitter the Python way. We will also focus on web development by using the famous Django framework to create an online game store. We will then create a web-based messenger using the new Nameko microservice framework. We will cover topics like authenticating users and, storing messages in Redis. By the end of the book, you will have gained hands-on experience in coding with Python. Style and approach With a hands-on approach, Python Programming Blueprints guides you through diverse real-life projects to get you started; it presents most aspects of the Python programming language gradually, going f... How to build useful, real-world applications in the Python programming languageKey FeaturesDeliver scalable and high-performing applications in Python.Delve into the great ecosystem of Python frameworks and libraries through projects that you will build with this book.This comprehensive guide will help you demonstrate the power of Python by building practical projects.Book DescriptionPython is a very powerful, high-level, object-oriented programming language. It's known for its simplicity and huge community support. Python Programming Blueprints will help you build useful, real-world applications using Python. In this book, we will cover some of the most common tasks that Python developers face on a daily basis, including performance optimization and making web applications more secure. We will familiarize ourselves with the associated software stack and master asynchronous features in Python. We will build a weather application using command-line parsing. We will then move on to create a Spotify remote control where we'll use OAuth and the Spotify Web API. The next project will cover reactive extensions by teaching you how to cast votes on Twitter the Python way. We will also focus on web development by using the famous Django framework to create an online game store. We will then create a web-based messenger using the new Nameko microservice framework. We will cover topics like authenticating users and, storing messages in Redis. By the end of the book, you will have gained hands-on experience in coding with Python.What you will learn[•] Learn object-oriented and functional programming concepts while developing projects[•] The dos and don'ts of storing passwords in a database[•] Develop a fully functional website using the popular Django framework[•] Use the Beautiful Soup library to perform web scrapping[•] Get started with cloud computing by building microservice and serverless applications in AWS[•] Develop scalable and cohesive microservices using the Nameko framework[•] Create service dependencies for Redis and PostgreSQLWho this book is forThis book is for software developers who are familiar with Python and want to gain hands-on experience with web and software development projects. A basic knowledge of Python programming is required. Annotation How to build useful, real-world applications in the Python programming languageKey Features Deliver scalable and high-performing applications in Python. Delve into the great ecosystem of Python frameworks and libraries through projects that you will build with this book. This comprehensive guide will help you demonstrate the power of Python by building practical projects. Book DescriptionPython is a very powerful, high-level, object-oriented programming language. It's known for its simplicity and huge community support. Python Programming Blueprints will help you build useful, real-world applications using Python. In this book, we will cover some of the most common tasks that Python developers face on a daily basis, including performance optimization and making web applications more secure. We will familiarize ourselves with the associated software stack and master asynchronous features in Python. We will build a weather application using command-line parsing. We will then move on to create a Spotify remote control where we'll use OAuth and the Spotify Web API. The next project will cover reactive extensions by teaching you how to cast votes on Twitter the Python way. We will also focus on web development by using the famous Django framework to create an online game store. We will then create a web-based messenger using the new Nameko microservice framework. We will cover topics like authenticating users and, storing messages in Redis. By the end of the book, you will have gained hands-on experience in coding with Python. What you will learn Learn object-oriented and functional programming concepts while developing projects The dos and don'ts of storing passwords in a database Develop a fully functional website using the popular Django framework Use the Beautiful Soup library to perform web scrapping Get started with cloud computing by building microservice and serverless applications in AWS Develop scalable and cohesive microservices using the Nameko framework Create service dependencies for Redis and PostgreSQLWho this book is forThis book is for software developers who are familiar with Python and want to gain hands-on experience with web and software development projects. A basic knowledge of Python programming is required How To Build Useful, Real-world Applications In The Python Programming Language Key Features Deliver Scalable And High-performing Applications In Python. Delve Into The Great Ecosystem Of Python Frameworks And Libraries Through Projects That You Will Build With This Book. This Comprehensive Guide Will Help You Demonstrate The Power Of Python By Building Practical Projects. Book Description Python Is A Very Powerful, High-level, Object-oriented Programming Language. It's Known For Its Simplicity And Huge Community Support. Python Programming Blueprints Will Help You Build Useful, Real-world Applications Using Python. In This Book, We Will Cover Some Of The Most Common Tasks That Python Developers Face On A Daily Basis, Including Performance Optimization And Making Web Applications More Secure. We Will Familiarize Ourselves With The Associated Software Stack And Master Asynchronous Features In Python. We Will Build A Weather Application Using Command-line Parsing. We Will Then Move On To Create A Spotify Remote Control Where We'll Use Oauth And The Spotify Web Api. The Next Project Will Cover Reactive Extensions By Teaching You How To Cast Votes On Twitter The Python Way. We Will Also Focus On Web Development By Using The Famous Django Framework To Create An Online Game Store. We Will Then Create A Web-based Messenger Using The New Nameko Microservice Framework. We Will Cover Topics Like Authenticating Users And, Storing Messages In Redis. By The End Of The Book, You Will Have Gained Hands-on Experience In Coding With Python. What You Will Learn Learn Object-oriented And Functional Programming Concepts While Developing Projects The Dos And Don'ts Of Storing Passwords In A Database Develop A Fully Functional Website Using The Popular Django Framework Use The Beautiful Soup Library To Perform Web Scrapping Get Started With Cloud Computing By Building Microservice And Serverless Applications In Aws Develop Scalable And Cohesive Microservices Using The Nameko Framework Create Service Dependencies For Redis And Postgresql Who This Book Is For This Book Is For Software Developers Who Are Familiar With Python And Want To Gain Hands-on Experience With Web And Software Development Projects. A Basic Knowledge Of Python Programming Is Required. Test your Python programming skills by solving real-world problems About This Book Access built-in documentation tools and improve your code. Discover how to make the best use of decorator and generator functions Enhance speed and improve concurrency by conjuring tricks from the PyPy project Who This Book Is For Whether you've been working with Python for a few years or you're a seasoned programmer, you'll have a lot of new tricks to walk away with. What You Will Learn Know the differences between .py and .pyc files Explore the different ways to install and upgrade Python packages Understand the working of the PyPI module that enhances built-in decorators See how coroutines are different from generators and how they can simulate multithreading Grasp how the decimal module improves floating point numbers and their operations Standardize sub interpreters to improve concurrency Discover Python's built-in docstring analyzer In Detail This book covers the unexplored secrets of Python, delve into its depths, and uncover its mysteries. You'll unearth secrets related to the implementation of the standard library, by looking at how modules actually work. You'll understand the implementation of collections, decimals, and fraction modules. If you haven't used decorators, coroutines, and generator functions much before, as you make your way through the recipes, you'll learn what you've been missing out on. We'll cover internal special methods in detail, so you understand what they are and how they can be used to improve the engineering decisions you make. Next, you'll explore the CPython interpreter, which is a treasure trove of secret hacks that not many programmers are aware of. We'll take you through the depths of the PyPy project, where you'll come across several exciting ways that you can improve speed and concurrency. Finally, we'll take time to explore the PEPs of the latest versions to discover some interesting hacks. Style and approach Recipe based approach where each problem is solved with the help of step by step instructions Annotation Test your Python programming skills by solving real-world problemsKey FeaturesAccess built-in documentation tools and improve your code. Discover how to make the best use of decorator and generator functionsEnhance speed and improve concurrency by conjuring tricks from the PyPy projectBook DescriptionThis book covers the unexplored secrets of Python, delve into its depths, and uncover its mysteries. You'll unearth secrets related to the implementation of the standard library, by looking at how modules actually work. You'll understand the implementation of collections, decimals, and fraction modules. If you haven't used decorators, coroutines, and generator functions much before, as you make your way through the recipes, you'll learn what you've been missing out on. We'll cover internal special methods in detail, so you understand what they are and how they can be used to improve the engineering decisions you make. Next, you'll explore the CPython interpreter, which is a treasure trove of secret hacks that not many programmers are aware of. We'll take you through the depths of the PyPy project, where you'll come across several exciting ways that you can improve speed and concurrency. Finally, we'll take time to explore the PEPs of the latest versions to discover some interesting hacks. What you will learnKnow the differences between .py and .pyc files Explore the different ways to install and upgrade Python packagesUnderstand the working of the PyPI module that enhances built-in decoratorsSee how coroutines are different from generators and how they can simulate multithreadingGrasp how the decimal module improves floating point numbers and their operationsStandardize sub interpreters to improve concurrencyDiscover Python's built-in docstring analyzerWho this book is forWhether you've been working with Python for a few years or you're a seasoned programmer, you'll have a lot of new tricks to walk away with Do you feel you've mastered the Python language and you know everything it takes to write applications that are a class apart? Well, you're in for a surprise! This book covers the darkest secrets of Python, delving into its depths and uncovering things you never would have thought could be done