New Feature of JUnit: Rules

Jens Schauder — Sun, 04 Oct 2009 10:12:20 +0000

card board box

I am always surprised how many unknown feature hide in a supposedly simple library. Todays example is JUnit. When inspecting the newest version (4.7) I noted an annotation I hadn’t noticed before: @Rule. WTF? I am looking at a testing framework and not at a rules engine, am I? So naturally I tried to find out what it is good for, and since not to much documentation is available my little research resulted in this blog post. Enjoy.

The purpose of the @Rule annotation is to mark public fields of a test class. These fields must be of type MethodRule, or an implementing class. Such MethodRules behave similar to a AOP aspects, of course without use of any AOP library and specialized for Tests. They can execute code before, after or instead of a test method. Example use cases listed in the release notes include:

Notification on tests
Setting up or tearing down resources, especially when they are used in multiple test classes
Special checks performed after every test, possibly causing a test to fail.
Making information about the test available inside the test

But ready made examples are boring, so I decided to try to implement something I was looking for all the time.

When writing acceptance tests with Fit or a similiar framework, you might end up with tons of failing tests, simply because the features aren’t implemented yet. But I don’t want this red lights hide a real red light, i.e. a test that worked before, but fails now. Therefore I’d like tests to get ignored just as if they where accordingly annotated, until they succeed for the first time. After that they should run (and possibly fail) just as a normal.

Let’s get started. First we need a little test case that can fail or succeed as we want. And this test case of course needs our annotated field:

public class DummyTest {

@Rule
public IgnoreLeadingFailure ilf = new IgnoreLeadingFailure();
@Test
public void testTest() {
assertTrue(false);
}
}

Please note that I stripped all package and import statements for brevity. As you can see I already named my MethodRule implementation IgnoreLeadingFailure. It needs some way to track which tests ran successfully at least once. Since I am lazy, I stored this information in a property file.

public class IgnoreLeadingFailure implements MethodRule {

private final static String PROPERTY_FILE_NAME = "activatedTests.properties";
private final static Properties activatedTests = new Properties();
static {
try {
activatedTests.load(new FileInputStream(PROPERTY_FILE_NAME));
} catch (IOException e) {
// actually this is to be expected on the first run
System.out.println("Couldn't load Properties from file" + e);
}
}
public Statement apply(final Statement base, final FrameworkMethod method,
final Object target) {
if (activatedTests.containsKey(getFullTestMethodName(method, target))) {
return base;
} else {
return new Statement() {
@Override
public void evaluate() throws Throwable {
try {
base.evaluate();
activateTest(getFullTestMethodName(method, target));
} catch (Throwable t) {
throw new AssumptionViolatedException(
"This test never succeeded before, and failed again with: "
+ t.toString());
}
}
private void activateTest(String fullTestMethodName) {
activatedTests.put(fullTestMethodName,
new SimpleDateFormat().format(new Date()));
try {
activatedTests.store(new FileOutputStream(
PROPERTY_FILE_NAME),
"tests that ran successfully at least once");
} catch (IOException io) {
System.out.println("failed to store properties" + io);
}
}
};
}
}
private String getFullTestMethodName(final FrameworkMethod method,
Object target) {
return target.getClass().getName() + " " + method.getName();
}
}

The intersting part is the single method contained in the MethodRule interface:

public Statement apply(final Statement base, final FrameworkMethod method, final Object target)

base is the object that encapsulates the execution of the test method. The purpose of apply(..) is to provide a Statement. This Statement will get executed (or in the lingo of the interface ‘evaluated’). You can return just , when you don't want to do anything, or more typically you'll return a Statement implementation, that wraps the Statement instance passed as a parameter.

method is the test method that will eventually get called and target is the object containing that method. Note that you normally do not execute the method, but use it mainly to get more information about the test case the MethodRule is applied to. With this information it should be easy to understand what my little class does: It checks if the test at hand was already executed successfully before. If this is the case, the same Statement is returned, that was passed as a parameter. Otherwise an anonymous implementation is returned which replaces any failure by a AssumptionViolatedException, and saves a successful test run in the property file. Note that the AssumptionViolatedException actually does not result in a ignored test, but gets displayed as a successful test, at least with the default runner, which doesn't really fit my needs, but one could easily fix that by a minor change in the runner but not without that change. So here is my opinion about this new JUnit feature: It certainly can come in handy for (integration) tests, where one tends to need a lot of resources that need quite some setup and/or tear down. Implementation of a MethodRule is fairly easy, although not exactly straight forward. The biggest problem is that it is next to impossible for an average java developer who doesn't know this feature to understand what is going on. The annotated field is easily missed, since in many cases it isn't used at all in the test class. Once more all this could be straight forward when java would support natively things like aspect oriented programming or open classes. If you want more information I suggest you drop by this blog post about 'Interceptors' which was the name for rules at the time of writing of that post. So, what do you think of this feature? Any ideas for features that could be implemented using rules?

Schauderhaft » Rule

New Feature of JUnit: Rules