Pro Python Best Practices : Debugging, Testing and Maintenance

Learn software engineering and coding best practices to write Python code right and error free. In this book you'll see how to properly debug, organize, test, and maintain your code, all of which leads to better, more efficient coding. Software engineering is difficult. Programs of any substantial length are inherently prone to errors of all kinds. The development cycle is full of traps unknown to the apprentice developer. Yet, in Python textbooks little attention is paid to this aspect of getting your code to run. At most, there is a chapter on debugging or unit testing in your average basic Python book. However, the proportion of time spent on getting your code to run is much higher in the real world. Pro Python Best Practices aims to solve this problem. You will: Learn common debugging techniques that help you find and eliminate errors Gain techniques to detect bugs more easily ver best="" practices="" to="" prevent="" bugscarry="" out="" automated="" testing="" discover="" problems="" faster Use best practices to maintain a project over a long time Learn techniques to keep your project under control.;Chapter 2: Exceptions in PythonExceptions Are Defects We Know About; Reading the Code; SyntaxError; Best Practices for Debugging SyntaxErrors; Examining the Error Message; The Error Type; The Error Description; The Traceback; Deduction; Catching Exceptions; Best Practices for Debugging IOErrors; Errors and Defects; Where Do Defects Come From?; The Correct Code; Best Practices; Chapter 3: Semantic Errors in Python; Comparing Expected and Factual Output; Defects; Defects in Variable Assignments; Multiple Initialization; Accidental Assignment; Accidental Comparison.;Overview of IPython CommandsExploring Namespaces; Exploring Namespaces with dir(); Exploring Namespaces of Objects; Exploring Attributes in a Python Program; Alternatives to dir in IPython; Namespace Mechanics; Python Uses Namespaces for Its Own Functions; Modifying a Namespace; Namespaces and Local Scope; Namespaces Are a Core Feature of Python; Using Self-Documenting Objects; Accessing Docstrings with help(); Object Summaries in IPython; Analyzing Object Types; Checking Object Identity; Checking Instances and Subclasses; Practical Use of Introspection; Finding Typos with Introspection.;Explain the Problem to Someone ElsePair Programming; Code Reviews; Reading; Cleaning Up; The Scientific Method and Other Best Practices; Best Practices; Chapter 5: Debugging with print Statements; Diagnose Whether Code Was Executed; Print the Content of Variables; Pretty-Printing Data Structures; Simplify Input Data; Start with Minimal Input; Gradually Add More Input Data; Switching print Output On and Off; Complete Code; Pros and Cons of Using print Statements; Best Practices; Chapter 6: Debugging with Introspection Functions; Explorative Coding in IPython; Exploring Files and Directories.;At a Glance; Contents; About the Author; About the Technical Reviewer; Acknowledgments; Preface; Chapter 1: Introduction; A Lesson in Humility; The Case for Best Practices in Python; The Origin of Best Practices; Hacking; Software Engineering; Agile; Software Craftsmanship; Who This Book Is For; What This Book Is About; Part 1: Debugging; Part 2: Automated Testing; Part 3: Maintenance; Further Benefits; The MazeRun Game; How to Use This Book?; Install Python 3; Install the Pygame Library; Install a Text Editor; Download the Source Code Examples; Part I: Debugging.;Wrong Variables in an ExpressionSwapped Literals in Expression; Defects in Indexing; Creating Wrong Indices; Using Wrong Indices; Defects in Control Flow Statements; Defects in Boolean Expressions; Defects with Indentation; Defects in Using Functions; Omitting a Function Call; Missing Return Statement; Not Storing the Return Value; Error Propagation; Best Practices; Chapter 4: Debugging with the Scientific Method; Applying the Scientific Method; Reproducing the Error; Automating the Error; Isolating the Defect; The Strip-Down Strategy; The Binary Search Strategy; Getting Help; Taking a Break. Contents at a Glance Contents About the Author About the Technical Reviewer Acknowledgments Preface Chapter 1: Introduction A Lesson in Humility The Case for Best Practices in Python The Origin of Best Practices Hacking Software Engineering Agile Software Craftsmanship Who This Book Is For What This Book Is About Part 1: Debugging Part 2: Automated Testing Part 3: Maintenance Further Benefits The MazeRun Game How to Use This Book? Install Python 3 Install the Pygame Library Install a Text Editor Download the Source Code Examples Part I: Debugging Chapter 2: Exceptions in Python Exceptions Are Defects We Know About Reading the Code SyntaxError Best Practices for Debugging SyntaxErrors Examining the Error Message The Error Type The Error Description The Traceback Deduction Catching Exceptions Best Practices for Debugging IOErrors Errors and Defects Where Do Defects Come From? The Correct Code Best Practices Chapter 3: Semantic Errors in Python Comparing Expected and Factual Output Defects Defects in Variable Assignments Multiple Initialization Accidental Assignment Accidental Comparison Wrong Variables in an Expression Swapped Literals in Expression Defects in Indexing Creating Wrong Indices Using Wrong Indices Defects in Control Flow Statements Defects in Boolean Expressions Defects with Indentation Defects in Using Functions Omitting a Function Call Missing Return Statement Not Storing the Return Value Error Propagation Best Practices Chapter 4: Debugging with the Scientific Method Applying the Scientific Method Reproducing the Error Automating the Error Isolating the Defect The Strip-Down Strategy The Binary Search Strategy Getting Help Taking a Break Explain the Problem to Someone Else Pair Programming Code Reviews Reading Cleaning Up The Scientific Method and Other Best Practices Best Practices Chapter 5: Debugging with print Statements Diagnose Whether Code Was Executed Print the Content of Variables Pretty-Printing Data Structures Simplify Input Data Start with Minimal Input Gradually Add More Input Data Switching print Output On and Off Complete Code Pros and Cons of Using print Statements Best Practices Chapter 6: Debugging with Introspection Functions Explorative Coding in IPython Exploring Files and Directories Overview of IPython Commands Exploring Namespaces Exploring Namespaces with dir() Exploring Namespaces of Objects Exploring Attributes in a Python Program Alternatives to dir in IPython Namespace Mechanics Python Uses Namespaces for Its Own Functions Modifying a Namespace Namespaces and Local Scope Namespaces Are a Core Feature of Python Using Self-Documenting Objects Accessing Docstrings with help() Object Summaries in IPython Analyzing Object Types Checking Object Identity Checking Instances and Subclasses Practical Use of Introspection Finding Typos with Introspection Combining Introspection Functions Introspection in Big and Small Programs Best Practices Chapter 7: Using an Interactive Debugger The Interactive Debugger ipdb Installing ipdb Starting the Debugger Running ipdb from the Command Line Starting ipdb from a Program Postmortem Debugging Launching the Debugger on Exceptions Fixing the Defect Commands at the Debugger Prompt Inspect Variables Evaluate Python Expressions Stepping Through Our Code Start Over Using Breakpoints Viewing and Deleting Breakpoints Conditional Breakpoints Configuring ipdb Example ipdb Session Adding a Game Control Function Stepping Through the Code Fixing the Defect It Is Working! Is the Program Without Defects Now? Other Debugging Tools pdb, the Python Debugger The PyCharm IDE ipdbplugin pudb wdb django-debug-toolbar cProfile Best Practices Part II: Automated Testing Chapter 8: Writing Automated Tests Installing py.test Writing a Test Function Running Tests Writing a Failing Test Making the Test Pass Passing Versus Failing Tests Writing Separate Test Functions Assertions Provide Helpful Output Testing for Exceptions Border Cases Complex Border Cases Benefits of Automated Testing Other Test Frameworks in Python unittest nose doctest Writing a __main__ block Best Practices Chapter 9: Organizing Test Data Using Fixtures The scope Parameter Test Parameterization Multiple Parameters Parametrized Fixtures Mocking Testing Output Files Cleaning Up After Tests Using Temporary Files Comparing Output Files with Test Data The filecmp Module The difflib Module Best Practices for Tests Involving Large Files Generating Random Test Data Where to Store Test Data? Test Data Modules Test Data Directories Test Databases Best Practices Chapter 10: Writing a Test Suite Test Modules Test Classes Refactoring Test Functions Fixtures in Test Classes How Do Tests Find the Tested Code? Multiple Test Packages Test Autodiscovery Executing a Test Suite Partial Execution Executing Test Modules and Packages Executing Test Classes Executing Single Tests Selecting Tests by Keywords Examining Failures Rerunning Tests Calculating Test Coverage A Test Suite Needs Maintenance Best Practices Chapter 11: Testing Best Practices Types of Automated Tests Unit Tests Integration Tests Acceptance Tests Regression Tests Performance Tests Performance Optimization The Test-First Approach Writing Tests Against a Specification Writing Tests Against Defects Test-Driven Development Advantages of Automated Testing Testing Saves Time Testing Adds Precision Testing Makes Collaboration Easier Limitations of Automated Testing Testing Requires Testable Code Testing Does Not Work Well for Projects Evolving Quickly Testing Does Not Prove Correctness Programs That Are Difficult to Test Random Numbers Graphical User Interfaces Complex or Large Output Concurrency Situations Where Automated Tests Fail Alternatives to Automated Testing Prototyping Code Reviews Checklists Processes Promoting Correctness Conclusions Best Practices Part III: Maintenance Chapter 12: Version Control Getting Started with git Creating a Repository Adding Files to a Repository Tracking Changes in Files Moving and Deleting Files Discarding Changes Navigating the History of Our Code Checking out Older Commits Traveling Back to the Most Recent Commit Publishing Code on GitHub Starting a Project on GitHub Using GitHub as a Single Contributor Working on Projects Started by Other People Projects with Multiple Contributors Merging Changes by Two People Pull Requests Development with Branches Merging Branches Configuring git Ignoring Files Global Settings Usage Examples Twenty Characters: A Small Project with Low Traffic Python: A Huge Project with Daily Commits grep: A Long-Term Project Other Version Control Systems Mercurial Subversion (SVN) Concurrent Versions Software (CVS) Bitbucket Sourceforge Best Practices Chapter 13: Setting Up a Python Project Creating a Project Structure with pyscaffold Installing pyscaffold Typical Directories in a Python Project Directories Created by pyscaffold The Main Python Package Directory The tests/ Directory The docs/ Directory The .git / Directory Directories Not Created by pyscaffold The bin/ Directory The Directories build/, dist/, and sdist/ The .hg/ Directory Data Directories Files Files Created by pyscaffold README.rst setup.py AUTHORS.rst LICENSE.rst MANIFEST.in versioneer.py requirements.txt .coveragerc .gitattributes and .gitignore Files Not Created by pyscaffold Setting the Version Number of Our Program Managing a Python Project Environment with virtualenv Installing virtualenv Connecting a Project to virtualenv Working with a virtualenv Project Installing Packages in virtualenv Leaving a virtualenv Session Configuring virtualenv Startup and Deactivation Setting the PYTHONPATH Variable Installing Pygame with virtualenv Best Practices Chapter 14: Cleaning Up Code Organized and Unorganized Code Software Entropy: Causes of Unorganized Code How to Recognize Unorganized Code? Readability Structural Weaknesses Redundancy Design Weaknesses Cleaning Up Python Instructions Place import Statements Together Place Constants Together Remove Unnecessary Lines Choose Meaningful Variable Names Idiomatic Python Code Refactoring Extract Functions Create a Simple Command-Line Interface Structuring Programs into Modules The Cleaned Code PEP8 and pylint Warning Messages Code Score Make It Work, Make It Right, Make It Fast Make It Work Make It Right Make It Fast Examples of Well-Organized Code Best Practices Chapter 15: Decomposing Programming Tasks Decomposing Programming Tasks Is Difficult A Process to Decompose Programming Tasks Write a User Story Add Details to the Description Acceptance Criteria Use Case Descriptions Check Nonfunctional Requirements Identify Problems Incomplete Information Domain Expertise Changing Existing Code Anticipating Future Change Decide on an Architecture Identify Program Components Implement Other Planning Tools The One-Page Project Plan Issue Trackers Kanban Best Practices Chapter 16: Static Typing in Python Weaknesses of Dynamic Typing Function Signatures Value Boundaries Semantic Meaning of Types Composite Types Is Stronger Typing Possible in Python? Assertions NumPy Databases Integrating C Code Cython Type Hints mypy Which Method of Type Control to Use? Best Practices Chapter 17: Documentation Who Do We Write Documentation For? Sphinx: A Documentation Tool for Python Setting Up Sphinx Files Created by Sphinx Building the Documentation Building HTML Documentation Building PDF Documentation Building EPUB Documentation Writing Documentation Directives Organizing Documents Code Examples Generating Documentation from Docstrings Doctests Configuring Sphinx Todo Entries Creating a Todo-List Conditional Building Changing the Look and Feel How to Write Good Documentation? Text Sections in Technical Documentation Summary Prerequisites and Installation Getting Started Cookbook Case Studies Technical Reference Design Documentation Legal Aspects Examples of Good Documentation Other Documentation Tools MkDocs Jupyter Notebooks Gitbook Read the Docs pydoc S5 pygments doctest PyPDF2 pandoc Best Practices Index Learn software engineering and coding best practices to write Python code right and error free. In this book you’ll see how to properly debug, organize, test, and maintain your code, all of which leads to better, more efficient coding. Software engineering is difficult. Programs of any substantial length are inherently prone to errors of all kinds. The development cycle is full of traps unknown to the apprentice developer. Yet, in Python textbooks little attention is paid to this aspect of getting your code to run. At most, there is a chapter on debugging or unit testing in your average basic Python book. However, the proportion of time spent on getting your code to run is much higher in the real world. Pro Python Best Practices aims to solve this problem. What You'll Learn Learn common debugging techniques that help you find and eliminate errors Gain techniques to detect bugs more easily discover best="" practices="" to="" prevent="" bugscarry="" out="" automated="" testing="" discover="" problems="" fasteruse="" maintain="" a="" project="" over="" l timeLearn techniques to keep your project under controlbr/uldivbWho This Book Is For/bbr/divdivbr/divdivExperienced Python coders from web development, big data, and more./divdivbr/divdivdiv/div Learn software engineering and coding best practices to write Python code right and error free. In this book you'll see how to properly debug, organize, test, and maintain your code, all of which leads to better, more efficient coding. Software engineering is difficult. Programs of any substantial length are inherently prone to errors of all kinds. The development cycle is full of traps unknown to the apprentice developer. Yet, in Python textbooks little attention is paid to this aspect of getting your code to run. At most, there is a chapter on debugging or unit testing in your average basic Python book. However, the proportion of time spent on getting your code to run is much higher in the real world. Pro Python Best Practices aims to solve this problem. What You'll Learn Learn common debugging techniques that help you find and eliminate errors Gain techniques to detect bugs more easily Who This Book Is For Experienced Python coders from web development, big data, and more.