More on JUnit Theories

Jens Schauder — Sun, 07 Feb 2010 13:02:21 +0000

In my last blog post, I described how to use JUnit Theories to create large amounts of test runs, with very limited amount of work, like so:

import static org.junit.Assume.assumeTrue;

@RunWith(Theories.class)
public class TheorieTest {
@DataPoint
public static String a = "a";
@DataPoint
public static String b = "bb";
@DataPoint
public static String c = "ccc";
@Theory
public void stringTest(String x, String y) {
assumeTrue(x.length() > 1);
System.out.println(x + " " + y);
}
}

The trick is simple to provide data points for every parameter type of the test method. The JUnit Theories Runner will call the test method with every possible combination of datapoints. If you think a little about it you will soon realize some of the limitations of this approach:

You’ll soon end up with lots of data point fields cluttering your code
Parameters of the same type will receive the same set of parameters, even when the usable range of inputs is completely different.

Fortunately the developers of JUnit provided really nice solutions to these problems.

Instead of specifying single data points, you can provide a full array of datapoints using the @Datapoints annotation, like so (add imports for good measure):

@RunWith(Theories.class)
public class TheorieTest {
@DataPoints
public static String[] a = { "a", "bb", "ccc" };
@DataPoints
public static Integer[] j = { 1, 2, 3 };
@Theory
public void someTest(String x, Integer y) {
assumeTrue(x.length() > 1);
System.out.println(x + " " + y);
}
}

This of course is much less verbose. Instead of an array you may provide a method returning an array, or at least it looks like this should be possible. But when I tried it JUnit seemed unable to handle the types correctly resulting in IllegalArgumentExceptions. Guess I’ll have to file a bug when finished with this article …

But we still need to take care of parameters which have the same type, but very different meaning and therefore different useful values. The clean OO way of doing things would be to get rid of the generic types like String and use stronger types like CreditCardNumber or Name instead. But then in a perfect world we wouldn’t need tests, because our programs wouldn’t contain any bugs to begin with. So lets try this instead (Again imports omitted):

@Retention(RetentionPolicy.RUNTIME)
@ParametersSuppliedBy(CreditCardSupplier.class)
public @interface AllCreditCards {}
//—————————————————————–
@Retention(RetentionPolicy.RUNTIME)
@ParametersSuppliedBy(NameSupplier.class)
public @interface AllNames {}
//—————————————————————–
public class CreditCardSupplier extends ParameterSupplier {
@Override
public List<PotentialAssignment> getValueSources(
ParameterSignature signature) {
ArrayList<PotentialAssignment> result = new ArrayList<PotentialAssignment>();
result.add(PotentialAssignment.forValue("Amex", "Amex"));
result.add(PotentialAssignment.forValue("Master", "Master"));
result.add(PotentialAssignment.forValue("Visa", "Visa"));
return result;
}
}
//—————————————————————–
public class NameSupplier extends ParameterSupplier {
@Override
public List<PotentialAssignment> getValueSources(
ParameterSignature signature) {
AllNames annotation = signature.getAnnotation(AllNames.class);
System.out.println("just wanted to show that I can access it "
+ annotation);
ArrayList<PotentialAssignment> result = new ArrayList<PotentialAssignment>();
result.add(PotentialAssignment.forValue("Alf", "Alf"));
result.add(PotentialAssignment.forValue("Willie", "Willie"));
result.add(PotentialAssignment.forValue("Tanner", "Tanner"));
result.add(PotentialAssignment.forValue("Cat", "Cat"));
return result;
}
}
//—————————————————————–
@RunWith(Theories.class)
public class SuppliedByTest {
@Theory
public void imagineThisIsATest(@AllCreditCards String x, @AllNames String y) {
System.out.println("consider " + x + " / " + y + " tested.");
}
@Theory
public void testIntegers(@TestedOn(ints = { 2, 3, 4, 7, 13, 23, 42 }) int i) {
System.out.println(i);
}
}

Wow, thats a lot of code. Just look at the last piece and see what appears in the console when we run it:

just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Amex / Alf tested.
consider Amex / Willie tested.
consider Amex / Tanner tested.
consider Amex / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Master / Alf tested.
consider Master / Willie tested.
consider Master / Tanner tested.
consider Master / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Visa / Alf tested.
consider Visa / Willie tested.
consider Visa / Tanner tested.
consider Visa / Cat tested.
2
3
4
7
13
23
42

Have a look at the row beginning with: “consider”. Obviously the Theory imagineThisIsATest gets fed with the values from the CreditCardSupplier and NameSupplier. The parameters and the ‘Suppliers’ are connected by the two annotations @AllNames and AllCreditCards. So whenever you have a parameter to a theory where the type alone is not sufficient for identifying the kind of values that should get used, you can simple create an annotation, which itself is annotated with a reference to a ParameterSupplier class and you are all set. You might think this is a lot of code for supplying a handful of parameters. You are right, but remember, that you can reuse your suppliers wherever you need names or credit card values in your tests.

Now let’s look at the first line of the output:
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
It simply shows of that you get access to the annotation (and actually the signature of the compete test method. This can be very useful, when you want your supplier to behave differently for different theories. Have a look at the NameSupplier above to see how this works.

JUnit actually comes with an example where this is used, and I demonstrated it with the other theory in the demonstration code above. The @TestedOn annotation takes an array of values to be used as data points for the annotated parameter.

Thats it for today. I hope the power of theories became obvious, as well as the power you have as a developer to extend that mechanism. Again be warned: All this nice stuff is in a package named experimental for good reason. If you use it, you might find bugs, and thing will likely change at least in name in an upcoming version. Taking about versions, I am using junit4.8.1 for the examples.

For next week the conclusion of the little series about JUnit theories is planned, with a few thoughts on use and danger of this kind of testing.

New Feature of JUnit: Theories

Jens Schauder — Sun, 31 Jan 2010 20:14:25 +0000

A couple of months ago I blogged about JUnit Rules, one of the new features in JUnit. While fooling around with JUnit Rules, I found a couple more features that you might be interested in. So here it comes: Theories! It turns out Theories are really a piece of cake. Try this:

import static org.junit.Assume.assumeTrue;

import org.junit.experimental.theories.DataPoint;
import org.junit.experimental.theories.Theories;
import org.junit.experimental.theories.Theory;
import org.junit.runner.RunWith;
@RunWith(Theories.class)
public class TheorieTest {
@DataPoint
public static String a = "a";
@DataPoint
public static String b = "bb";
@DataPoint
public static String c = "ccc";
@Theory
public void stringTest(String x, String y) {
assumeTrue(x.length() > 1);
System.out.println(x + " " + y);
}
}

When you run this you’ll get this as an output (and one successfull test):

bb a
bb bb
bb ccc
ccc a
ccc bb
ccc ccc

So what is going on? The first thing to note is that this test is executed by a specialized runner of type Theories. This runner executes all the public methods annotated with @Theory. Differently from normal tests theories have parameters. In order to fill these parameters with values, the Theories runner uses all the public fields of matching Type and annotated with @DataPoint. When you take a look at the output, it should be obvious why this is more powerful then parameterized Tests: Every combination of values is tried, thus with e.g. 4 parameters with 4 distinct values each, you end up with 256 test runs.

The idea is to specify a theory about the object under test, that holds for a large class of states and parameters. Then you provide those as parameters to the test method which will test the theory.

A probably very common case is that a theory is known not to be valid for certain cases. You can exclude these from a test using the Assume class. If an assumption doesn’t hold, the test is silently ignored. This is used in the example above to prevent ‘a’ to be used as a first parameter.

There are some more tweaks to Theories in JUnit, which I will cover in a later blog. Until then enjoy the new feature. But keep in mind the package in which theories reside up to now: org.junit.experimental.theories so I’d expect some changes in the API and at least a change in package name.

Schauderhaft » theory

More on JUnit Theories

New Feature of JUnit: Theories