The trick is simple to provide data points for every parameter type of the test method. The JUnit Theories Runner will call the test method with every possible combination of datapoints. If you think a little about it you will soon realize some of the limitations of this approach:

• You’ll soon end up with lots of data point fields cluttering your code
• Parameters of the same type will receive the same set of parameters, even when the usable range of inputs is completely different.

Fortunately the developers of JUnit provided really nice solutions to these problems.

Instead of specifying single data points, you can provide a full array of datapoints using the @Datapoints annotation, like so (add imports for good measure):

This of course is much less verbose. Instead of an array you may provide a method returning an array, or at least it looks like this should be possible. But when I tried it JUnit seemed unable to handle the types correctly resulting in IllegalArgumentExceptions. Guess I’ll have to file a bug when finished with this article …

But we still need to take care of parameters which have the same type, but very different meaning and therefore different useful values. The clean OO way of doing things would be to get rid of the generic types like String and use stronger types like CreditCardNumber or Name instead. But then in a perfect world we wouldn’t need tests, because our programs wouldn’t contain any bugs to begin with. So lets try this instead (Again imports omitted):

Wow, thats a lot of code. Just look at the last piece and see what appears in the console when we run it:

just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Amex / Alf tested.
consider Amex / Willie tested.
consider Amex / Tanner tested.
consider Amex / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Master / Alf tested.
consider Master / Willie tested.
consider Master / Tanner tested.
consider Master / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Visa / Alf tested.
consider Visa / Willie tested.
consider Visa / Tanner tested.
consider Visa / Cat tested.
2
3
4
7
13
23
42

Have a look at the row beginning with: “consider”. Obviously the Theory imagineThisIsATest gets fed with the values from the CreditCardSupplier and NameSupplier. The parameters and the ‘Suppliers’ are connected by the two annotations @AllNames and AllCreditCards. So whenever you have a parameter to a theory where the type alone is not sufficient for identifying the kind of values that should get used, you can simple create an annotation, which itself is annotated with a reference to a ParameterSupplier class and you are all set. You might think this is a lot of code for supplying a handful of parameters. You are right, but remember, that you can reuse your suppliers wherever you need names or credit card values in your tests.

Now let’s look at the first line of the output:
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
It simply shows of that you get access to the annotation (and actually the signature of the compete test method. This can be very useful, when you want your supplier to behave differently for different theories. Have a look at the NameSupplier above to see how this works.

JUnit actually comes with an example where this is used, and I demonstrated it with the other theory in the demonstration code above. The @TestedOn annotation takes an array of values to be used as data points for the annotated parameter.

Thats it for today. I hope the power of theories became obvious, as well as the power you have as a developer to extend that mechanism. Again be warned: All this nice stuff is in a package named experimental for good reason. If you use it, you might find bugs, and thing will likely change at least in name in an upcoming version. Taking about versions, I am using junit4.8.1 for the examples.

For next week the conclusion of the little series about JUnit theories is planned, with a few thoughts on use and danger of this kind of testing.

When you run this you’ll get this as an output (and one successfull test):

bb a
bb bb
bb ccc
ccc a
ccc bb
ccc ccc

So what is going on? The first thing to note is that this test is executed by a specialized runner of type Theories. This runner executes all the public methods annotated with @Theory. Differently from normal tests theories have parameters. In order to fill these parameters with values, the Theories runner uses all the public fields of matching Type and annotated with @DataPoint. When you take a look at the output, it should be obvious why this is more powerful then parameterized Tests: Every combination of values is tried, thus with e.g. 4 parameters with 4 distinct values each, you end up with 256 test runs.

The idea is to specify a theory about the object under test, that holds for a large class of states and parameters.  Then you provide those as parameters to the test method which will test the theory.

A probably very common case is that a theory is known not to be valid for certain cases. You can exclude these from a test using the Assume class. If an assumption doesn’t hold, the test is silently ignored. This is used in the example above to prevent ‘a’ to be used as a first parameter.

There are some more tweaks to Theories in JUnit, which I will cover in a later blog. Until then enjoy the new feature. But keep in mind the package in which theories reside up to now: org.junit.experimental.theories so I’d expect some changes in the API and at least a change in package name.

Fireworks

In a couple of days I’ll start working on a new project. Actually it is an project that I worked one or two years ago. I think I did decent job last time. But there is always lots of room for improvement. So today I want to list a couple of things I want to do bettern than the last time.

More Tests – When working on that project I was the only person with real experience in automated testing. I am proud that we had a test suite and that this test suite still exists and put to good use and actually grew during the last months. Yet I consider the coverage of the tests not sufficient by far. So for every code I write, I will add an extensive set of test. I’ll also add tests for existing code, when refactoring.

Earlier Tests – One reason we didn’t produce as many tests as I would wish today is that we didn’t do TDD. It probably was a correct decission the last time, since it would have been just to much. But this time I will practice TDD as good as I can.

Cleaner Code – One of the first tests I wrote back then was a test to ensure that we don’t have circular dependencies. We succeeded in this, but still parts of the system a pretty tangled. We gonna improve that. Appart from the knowledge I took from Clean Code I’ll plan to use the toxicity metric as an important tool for that.

Dependency Injection – We considered using Spring and decided against it, because we already had about 20 libraries we hardly knew in the project. Again at that time it was a reasonable decision. But in the meantime I learned what we could have gained by using Spring. I’ll propose using it, but even when the team declines again, I’ll be able to use the concept in order to improve the code base.

So these are the things I want to improve on in the next project. What are the things you want to improve on in your next project?

card board box

The purpose of the @Rule annotation is to mark public fields of a test class. These fields  must be of type MethodRule, or an implementing class. Such MethodRules behave similar to a AOP aspects, of course without use of any AOP library and specialized for Tests. They can execute code before, after or instead of a test method.  Example use cases listed in the release notes include:

• Notification on tests
• Setting up or tearing down resources, especially when they are used in multiple test classes
• Special checks performed after every test, possibly causing a test to fail.
• Making information about the test available inside the test

But ready made examples are boring, so I decided to try to implement something I was looking for all the time.

When writing acceptance tests with Fit or a similiar framework, you might end up with tons of failing tests, simply because the features aren’t implemented yet. But I don’t want this red lights hide a real red light, i.e. a test that worked before, but fails now. Therefore I’d like tests to get ignored just as if they where accordingly annotated, until they succeed for the first time. After that they should run (and possibly fail) just as a normal.

Let’s get started. First we need a little test case that can fail or succeed as we want. And this test case of course needs our annotated field:

Please note that I stripped all package and import statements for brevity. As you can see I already named my MethodRule implementation IgnoreLeadingFailure. It needs some way to track which tests ran successfully at least once. Since I am lazy, I stored this information in a property file.

The intersting part is the single method contained in the MethodRule interface:

public Statement apply(final Statement base, final FrameworkMethod method, final Object target)

base is the object that encapsulates the execution of the test method. The purpose of apply(..) is to provide a Statement. This Statement will get executed (or in the lingo of the interface ‘evaluated’). You can return just , when you don't want to do anything, or more typically you'll return a Statement implementation, that wraps the Statement instance passed as a parameter.