Nose timelimit

17 December 2013 (updated 17 February 2016)

I think everyone hates slow test suites but sometimes there's too much effort to make it fast. In some situations running your test database in-memory will not yield great improvement because you're probably doing the tests wrong :)

But you have too many of them, you have mixed fast and slow tests and tagging the tests with attributes isn't going to cut it - you ain't gonna manually go over each of those 2000 tests.

With nose-timelimit you can get from this:

test_activate_on_usr2 (test_manhole.ManholeTestCase) ... (11.9111 seconds) ok
test_activate_on_with_oneshot_on (test_manhole.ManholeTestCase) ... (0.2713 seconds) ok
test_auth_fail (test_manhole.ManholeTestCase) ... (0.3722 seconds) ok
test_exit_with_grace (test_manhole.ManholeTestCase) ... (0.6874 seconds) ok
test_fail_to_cry (test_manhole.ManholeTestCase) ... (0.0178 seconds) ok
test_interrupt_on_accept (test_manhole.ManholeTestCase) ... (2.2326 seconds) ok
test_oneshot_on_usr2 (test_manhole.ManholeTestCase) ... (11.8544 seconds) ok
test_oneshot_on_usr2_error (test_manhole.ManholeTestCase) ... (12.1931 seconds) ok
test_signalfd_weirdness (test_manhole.ManholeTestCase) ... (0.7931 seconds) ok
test_signalfd_weirdness_negative (test_manhole.ManholeTestCase) ... (1.6042 seconds) ok
test_simple_r01 (test_manhole.ManholeTestCase) ... (0.6593 seconds) ok
test_simple_r02 (test_manhole.ManholeTestCase) ... (0.9883 seconds) ok
test_simple_r03 (test_manhole.ManholeTestCase) ... (1.3314 seconds) ok
test_simple_r04 (test_manhole.ManholeTestCase) ... (1.6603 seconds) ok
test_simple_r05 (test_manhole.ManholeTestCase) ... (2.0033 seconds) ok
test_simple_r06 (test_manhole.ManholeTestCase) ... (2.3323 seconds) ok
test_simple_r07 (test_manhole.ManholeTestCase) ... (2.6754 seconds) ok
test_simple_r08 (test_manhole.ManholeTestCase) ... (2.9923 seconds) ok
test_simple_r09 (test_manhole.ManholeTestCase) ... (3.3474 seconds) ok
test_simple_r10 (test_manhole.ManholeTestCase) ... (3.4603 seconds) ok
test_with_fork (test_manhole.ManholeTestCase) ... (2.2701 seconds) ok
test_with_forkpty (test_manhole.ManholeTestCase) ... (2.1426 seconds) ok

----------------------------------------------------------------------
Ran 22 tests in 67.805s

OK

to this:

test_activate_on_usr2 (test_manhole.ManholeTestCase) ... SKIP: Last run took 11.6118 seconds. Too slow!
test_activate_on_with_oneshot_on (test_manhole.ManholeTestCase) ... (0.2684 seconds) ok
test_auth_fail (test_manhole.ManholeTestCase) ... (0.3813 seconds) ok
test_exit_with_grace (test_manhole.ManholeTestCase) ... (0.6571 seconds) ok
test_fail_to_cry (test_manhole.ManholeTestCase) ... (0.0162 seconds) ok
test_interrupt_on_accept (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.2251 seconds. Too slow!
test_oneshot_on_usr2 (test_manhole.ManholeTestCase) ... SKIP: Last run took 11.7995 seconds. Too slow!
test_oneshot_on_usr2_error (test_manhole.ManholeTestCase) ... SKIP: Last run took 11.3974 seconds. Too slow!
test_signalfd_weirdness (test_manhole.ManholeTestCase) ... (0.7740 seconds) ok
test_signalfd_weirdness_negative (test_manhole.ManholeTestCase) ... SKIP: Last run took 1.5585 seconds. Too slow!
test_simple_r01 (test_manhole.ManholeTestCase) ... (0.6602 seconds) ok
test_simple_r02 (test_manhole.ManholeTestCase) ... (0.9946 seconds) ok
test_simple_r03 (test_manhole.ManholeTestCase) ... SKIP: Last run took 1.3252 seconds. Too slow!
test_simple_r04 (test_manhole.ManholeTestCase) ... SKIP: Last run took 1.5586 seconds. Too slow!
test_simple_r05 (test_manhole.ManholeTestCase) ... SKIP: Last run took 1.9972 seconds. Too slow!
test_simple_r06 (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.1826 seconds. Too slow!
test_simple_r07 (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.6691 seconds. Too slow!
test_simple_r08 (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.8067 seconds. Too slow!
test_simple_r09 (test_manhole.ManholeTestCase) ... SKIP: Last run took 3.3411 seconds. Too slow!
test_simple_r10 (test_manhole.ManholeTestCase) ... SKIP: Last run took 3.6106 seconds. Too slow!
test_with_fork (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.3372 seconds. Too slow!
test_with_forkpty (test_manhole.ManholeTestCase) ... SKIP: Last run took 2.1546 seconds. Too slow!

----------------------------------------------------------------------
Ran 22 tests in 3.756s

OK (SKIP=15)

You can use nose-timelimit to get some quick test feedback - this is especially useful when you refactor code or otherwise make changes that might break more than 1 test. This implies that not all the tests are slow :)

Note: the results are colored with another very nice plugin: yanc.

This entry was tagged as nose python testing