in sw development

Debugging Python programs

There are different ways to debug Python programs, from printing information to the console using basic print() function to using a full-fledged debugger. In this article we will have a look at some basic tools that we can use, including less known tools like PySnooper or stackprinter.

I have created an accompanying repository python-debugging with all the examples, so you can check it out and try all the tools yourself. All examples follow a simple for loop where we are trying to see what is happening inside of it at every iteration. Please make sure that you follow installation and usage instruction in the README.

Debugging with f-strings

Since print() function is the most used way to debug Python applications during development, let’s start here. f-strings (formatted string literals) are available from Python 3.6 and they are arguably the most intuitive way to quickly include variables in our strings. Therefore it makes perfect sense to use them together with print()to print our data during execution of the program:

In Python 3.8, the f-string syntax will be enhanced with = specifier to make the debugging syntax even shorter:

The output will be:

Link to the example: Debugging with f-strings.

Debugging with PySnooper

What if we want to still use the same technique, printing information about our data to the console, but we don’t want to do that manually? PySnooper is a very neat tool to do just that. With a simple import and an annotation, we will get an extensive amount of information to our console. PySnooper will print the state of the program after each executed line of code from the annotated function:

The PySnooper will output this for us automatically:

There is a lot of options available, but I will mention just one thing here. Instead of using an annotation to mark a whole function, PySnooper’s pysnooper.snoop() is both a decorator and a context manager with which it is easy to debug just a small amount of code (and we don’t even have to have any functions defined):

Link to the example: Debugging with PySnooper.

Debugging with logging

Staying still within the printing realm of debugging, we can step up the game with a logging library. We can of course use any of the logging libraries available for Python, but I chose to use logging module from the standard library as it is already pre-installed.

Using a logging library is useful for debugging applications in production, and there are a lot of options to configure. In our example we will print some additional information (function name, line number), use different importance level of the message (debug, error) and print our messages to the console as well as to a file on the disk, to demonstrate at least some of the advantages over normal print().

The output from such a configured logger will be:

Link to the example: Debugging with logging module.

Debugging with breakpoint() and pdb

Python comes with its own Python debugger called pdb. As pdb is a proper debugger, we can do many things with it. In this example we will use a built-in function breakpoint() to stop the execution of our program inside the loop and then investigate our variables as we go through our loop. Let’s have a look at the code:

When this code runs on the command line, the program stops on the breakpoint line and an interactive pdb session is opened for us. It accepts commands that are executed by hitting enter as is normal on the command line. We can investigate the variables (i, processed_word) just by typing their name and hitting enter. To continue the execution and move to the next iteration, we can use command named continue.

I am including a record of such a session where I examine both of our variables at each step of the iteration:

Link to the example: Debugging with breakpoint() and pdb.

Debugging with VSCode

Very nice and comfortable debugging can be achieved with Visual Studio Code and Python extension.

Before we can use the debugging feature, we need to make sure that the Python path is correctly set for the project in VSCode. VSCode shows the Python version in the lower left corner and you can easily change this path just by clicking on it.

When this is done, we can go to the Debug panel and create our debug configuration. VSCode makes it easy to create a configuration for Flask, Django and Pyramid frameworks, but for our example, we will choose a simple Python file option.

The only thing left is to set our breakpoints. This is done by clicking on the left side of the line number on which we want to stop the program’s execution. Now when we run our debug session with the green play arrow, the program is executed and stopped on the marked line.

The left panel offers us a view of our variables and a new floating menu allows us to control the execution of the program.

Link to the example: Debugging with VSCode.

Debugging with stackprinter

stackprinter is a library that can provide nicer stack traces for our exceptions. Let’s setup stackprinter and produce an exception:

When we run this, we will see the stack trace in the stackprinter’s own format instead of the default stack trace output:

Link to the example: Debugging with stackprinter.

And this is all for now! Happy debugging!


Loading Likes...

Write a Comment

Comment

  1. Great list Petr!

    Have you tried debugging with other tools, to explore the side effects of code and its interactions, not just the line-by-line progress? Depends on the code of course, but for larger applications or applications with complex interactions it can be helpful to be able to take a step back and get an overview.

    I’ve just released an open-source tool to do exactly this for HTTP & HTTPS which you might find useful – check it out at https://httptoolkit.tech/view/python/ if that sounds interesting.

Check out Contact Cache, a contact and relationship manager with client-side encryption!