The most underrated feature in Python 3

Sun 03 August 2014

One of my pet peeves in Python is about reraising exceptions. This was incredibly frustrating till Python 3.

Few examples where reraising is needed:

  • Libraries that abstract the same task over different implementations. Example: a protocol library abstracting different transports. The transports don't raise the same exceptions (and not in the same way) - so it's a good idea to unify everything.
  • Dealing with broken libraries. At some point you may have to use library has bugs or it's badly designed: it will raise the wrong errors.

The traceback issue*

Imagine we have two modules: foo and bar. bar is using foo to perform something, among many other things - so it's a good idea to reduce and unify all the possible exceptions bar could raise. This means wrapping the exceptions raised by foo in a common set of exceptions (defined in bar).

Most of the time we would have this sort of code:

foo.py

class FooException(Exception):
    pass

def a():
    b()

def b():
    c()

def c():
    raise FooException("There's some problem ...")

bar.py

import foo

class BarException(Exception):
    pass

def a():
    b()

def b():
    c()

def c():
    try:
        foo.a()
    except foo.FooException as e:
        raise BarException(e)

Unfortunately the traceback would look like this on Python 2:

Traceback (most recent call last):
  File "bar.py", line 19, in <module>
    a()
  File "bar.py", line 7, in a
    b()
  File "bar.py", line 10, in b
    c()
  File "bar.py", line 16, in c
    raise BarException(e)
__main__.BarException: There's some problem ...

You can see the frames from foo are missing, leaving us with too little context to figure out what's going on.

To fix this we could do one of these:

  • Just use a bare raise. But then we don't wrap the exceptions anymore - leaving lots of possible exception types for the user of bar to handle.

  • Log the original exception, so the traceback with the frames from foo` is at least in the logs. This is obviously not optimal as the logging might be misconfiguration.

  • Raise the new exception with the original traceback:

    raise BarException, BarException(e), sys.exc_info()[2]
    

    Now we get this pretty good traceback:

    Traceback (most recent call last):
      File "bar.py", line 20, in <module>
        a()
      File "bar.py", line 8, in a
        b()
      File "bar.py", line 11, in b
        c()
      File "bar.py", line 15, in c
        foo.d()
      File "foo.py", line 5, in d
        e()
      File "foo.py", line 8, in e
        f()
      File "foo.py", line 11, in f
        raise FooException("There's some problem ...")
    __main__.BarException: There's some problem ...
    

    Unfortunately this is not very popular - to my frustration, virtually no one does it properly.

Python 3 is much better here, it doesn't put the burden of being traceback-aware on developers.

Going back to the original raise BarException(e), it would look like this on Python 3:

Traceback (most recent call last):
  File "bar.py", line 15, in c
    foo.d()
  File "foo.py", line 5, in d
    e()
  File "foo.py", line 8, in e
    f()
  File "foo.py", line 11, in f
    raise FooException("There's some problem ...")
foo.FooException: There's some problem ...

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bar.py", line 20, in <module>
    a()
  File "bar.py", line 8, in a
    b()
  File "bar.py", line 11, in b
    c()
  File "bar.py", line 17, in c
    raise BarException(e)
__main__.BarException: There's some problem ...

This is one of the many improvements from PEP-3134 [*] and it makes debugging much easier, especially for novices. If you haven't tried it yet you should really try Python 3 - it has very many small improvements that make development better.

[*]See Implicit Exception Chaining in PEP-3134.

This entry was tagged as python

blog comments powered by Disqus