The most underrated feature in Python 3

03 August 2014 (updated 16 June 2019)

Note

There's a nice video version of this article.

Courtesy of Webucator (they do Python training).

One of my pet peeves in Python is about reraising exceptions. This was incredibly frustrating till Python 3.

Few examples where reraising is needed:

  • Libraries that abstract the same task over different implementations. Example: a protocol library abstracting different transports. The transports don't raise the same exceptions (and not in the same way) - so it's a good idea to unify everything.
  • Dealing with broken libraries. At some point you may have to use library has bugs or it's badly designed: it will raise the wrong errors.

The traceback issue *

Imagine we have two modules: foo and bar. bar is using foo to perform something, among many other things - so it's a good idea to reduce and unify all the possible exceptions bar could raise. This means wrapping the exceptions raised by foo in a common set of exceptions (defined in bar).

Most of the time we would have this sort of code:

foo.py *

class FooException(Exception):
    pass

def d():
    e()

def e():
    f()

def f():
    raise FooException("There's some problem ...")

bar.py *

import foo

class BarException(Exception):
    pass

def a():
    b()

def b():
    c()

def c():
    try:
        foo.d()
    except foo.FooException as e:
        raise BarException(e)

Unfortunately the traceback would look like this on Python 2:

Traceback (most rTraceback (most recent call last):
  File "<string>", line 1, in <module>
  File "bar.py", line 7, in a
    b()
  File "bar.py", line 10, in b
    c()
  File "bar.py", line 16, in c
    raise BarException(e)
bar.BarException: There's some problem ...

You can see the frames from foo are missing, leaving us with too little context to figure out what's going on.

To fix this we could do one of these:

  • Just use a bare raise. But then we don't wrap the exceptions anymore - leaving lots of possible exception types for the user of bar to handle.

  • Log the original exception, so the traceback with the frames from foo is at least in the logs. This is obviously not optimal as the logging might be misconfiguration.

  • Raise the new exception with the original traceback, instead of raise BarException(e) have:

    raise BarException, BarException(e), sys.exc_info()[2]
    

    Now we get this pretty good traceback:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "bar.py", line 7, in a
        b()
      File "bar.py", line 10, in b
        c()
      File "bar.py", line 14, in c
        foo.d()
      File "foo.py", line 5, in d
        e()
      File "foo.py", line 8, in e
        f()
      File "foo.py", line 11, in f
        raise FooException("There's some problem ...")
    bar.BarException: There's some problem ...
    

    Unfortunately this is not very popular - to my frustration, virtually no one does it properly.

Python 3 is much better here, it doesn't put the burden of being traceback-aware on developers.

Going back to the original raise BarException(e), it would look like this on Python 3:

Traceback (most recent call last):
  File "bar.py", line 14, in c
    foo.d()
  File "foo.py", line 5, in d
    e()
  File "foo.py", line 8, in e
    f()
  File "foo.py", line 11, in f
    raise FooException("There's some problem ...")
foo.FooException: There's some problem ...

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "bar.py", line 7, in a
    b()
  File "bar.py", line 10, in b
    c()
  File "bar.py", line 16, in c
    raise BarException(e)
bar.BarException: There's some problem ...

This is one of the many improvements from PEP-3134 [*] and it makes debugging much easier, especially for novices. If you haven't tried it yet you should really try Python 3 - it has very many small improvements that make development better.

[*]See Implicit Exception Chaining in PEP-3134.

This entry was tagged as debugging python