On handling exceptions in happy path tests

I'd like to start this post with a reflection on the goal of unit testing.

The goal of unit tests is to validate the single, precise behavior of a block of code. This indicates that a test's value is directly dependent on its ability to spot deviations in that behavior.

So lets look at a simple test, the likes of which I have seen before:

void testSomeBizareCondition() {
 try {
  op.someOperartionThatShouldBeOK();
  assertEquals("abc", op.getValue());
 } catch (NullPointerException e) {
  fail("oops");
 } catch (SomeOperationException e) {
  fail("oops");
 }
}

What is wrong with this test? If the test should fail with an "oops" there is no way to differentiate between the two error conditions. Instead of reading the test report logs and quickly correcting the code, some developers will find themselves debugging the tests. The answer to that is not to change the failure messages for each catch, because this would lead us down the path of catching every imaginable exception type.

Another reason not to favor catching of individual exception types is illustrated bellow:

void testSomeBizareConditionThrowsAnOperationException() {
 try {
  op.someOperartionThatShouldBeOK();
  fail("oops");
 } catch (NullPointerException e) {
  fail("oops");
 } catch (SomeOperationException e) {
 }
}

At a glance, can you tell what the goal of the test is? Maybe you can, but it isn't obvious, and you shouldn't have to thing about it. This test is intended to validate that SomeOperationException is thrown during test execution. Unfortunately this intention is not evident.

In order to illustrate the solution to both of the above tests, lets look at the basic concept. Bellow is a test that represents a simplified version of the first example.

void testSomeOptimisticCondition() {
 try {
  op.someOperationThatShouldBeOK();
  assertEquals("abc", op.getValue());
 } catch (Trowable th) {
  fail("bad");
 }
}

This example still masks the failure reason, the cause of exception is not in evidence. The stack trace and exception details go a long way to facilitating a quick code correction.

A second problem with this test is that it masks the details of the assertion failure. This could be corrected by changing the catch type from Throwable to Exception. This is because an assertion failure is expressed in the form of an exception.

The solution

Its hard to imagine, but sometimes the best thing to do is let the exception propagate out of the test. For unchecked exceptions this is easy, the test will fail, and the cause will be plain as day in the test output.

For checked exceptions this is a little more complicated, but only slightly. Let the test method throw a generic exception.

void testSomeOptimisticCondition() throws Exception {
 op.someOperationThatShouldBeOK();
 assertEquals("abc", op.getValue());
}

Not only does the test now guaranty that all failure conditions are now explicit, it makes the test a lot clearer.

Test cleanup

Now all this discussion on removing clutter may well have some readers thinking: "yeah, but I use a catch to cleanup after the test". It is true that some tests require the cleaning of external resources. The problem with doing so in a try/catch is that it can complicate test logic. There are two viable solutions, one is to use a try/finally, putting the cleanup in the finally, and another is to put the logic in the tear down method. I tend to favor the tear-down, because it removes clutter from the test method.

Test should always fail fast. So how do we handle exceptions in happy-path tests? We don't.