Long Descriptive Name: For example returnsNullWhenCalledOnEmptyList is a nice name for a test

Simple Clean Structure: A test performs necessary setup, executes the method to be tested and performs its assertions. Not more not less

Fast: one thousand tests should run in less than a minute. This allows an extensive test suite to run often during development. If your tests are slower, you might be doing integration tests.

Test a Single Feature or behavior: This is sometimes phrased as ‘one assertion’, which I consider misleading. If you need multiple assertion in order to assert one behavior thats OK, although you might consider a new assert method for that.

Reliable: For the same code base a test should produce the same result every time. A test that fails every 10th time will fail for a real bug and you won’t know it.

Stable: The test only fails, when the code under test doesn’t behave as expected, not because anything else changed.

Easy to Understand: Just as normal code, the intention of a test should be easy to understand. Names help with this. Extracting setup code in a properly named method helps as well.

Should Not Use Production Classes, Except The Class Under Test: This helps with the ‘Stable’ and the ‘Fast’ property. It’s ok to use simple classes like String or Integer.

Don’t Try to Squeeze All Tests for a Class in a Single Test Class: For many classes you will have more then one test class. The different test classes will probably differ in the setup needed for the contained tests.

Independent: No Test should depend on any other Test or on the order of execution.

Let’s start with the well known LazyInitializationException. When you search for this one in the Internet you end up with scenarios like this one:

You load A, close the session you  used and try to access a property of A which gets loaded lazily resulting in a LazyInitializationException. That one is well described, including strategies to avoid it, which basically consist of not closing the session to early.

But there are more hideous scenarios. Assume the classes Mom, Dad and Kid are mapped as one would expect, with references from Mom and Dad to Kid. Now consider the following piece of code:

Depending on your exact Mapping you might get a LazyInitializationException in the last line, although the session is still open. The problem here is that when dad got evicted the kid got evicted as well. And since Mom has a reference to the exact same object, that object isn’t attached to any session anymore. In order to see this effect you’ll have to use mappings like this:

The NonUniqueObjectException gets caused by what could be considered the reverse process. Examine the following piece of code:

The first method call loads a Dad object including Kid from a different session. Then from the ‘main’ session Mom gets loaded including kid. When we attach the Dad object to the session it tries to attach its version of the Kid as well, which collides with the Kid already present in the session.

So how can we avoid these kind of problem? I see several options:

• Don’t use Cascade. Of course this means much more effort on your side in many cases, since you have to carefully track what you need to attach or evict from a session.
• Don’t use evict and don’t reattach existing instances to sessions. I guess this should work in many cases. But I have seen cases where this approach would have caused a large overhead, because reloading complex object graphs.
• Carefully design your cascading boundaries. A look at the Aggregate Root from DDD might help here. An Aggregate Root is a single class which works as a gate keeper (did anybody say facade) to a cluster of classes related to the Aggregate Root. When all the Hibernate related actions go against that Aggregate Root, and cascade from there to the content and only the content of the Aggregate Root you should be safe. All entities outside an Aggregate Root have to use the Aggregate Root or Repositories to gain access. This needs some bookkeeping. But when you do it properly it might actually improve the over all design of your code. Please note that all this is a thought experiment on my side, and I can’t promise that it actually works as intended, but I think it should.

If you want the complete source code for the example, you can download this source.zip file.

Assume you have a fairly complex select statement operating on some serious tables (tens of millions of rows). The customer complains the SQL statement is to slow. What options would you consider for tuning such a select statement? If it matters assume it is an Oracle database.

Just in case you want to apply for a job at a LINEAS here is, what I would consider a damn good answer:

First I check if the SQL statement is really slow, for that I’ll execute the SQL statement with a low level tool (SQL Plus, SQL Developer …) on the database. I fetch the same amount of rows, as the customer (or his application) does. I do that on the DB machine or one a machine with very good connectivity to the database, in order to take network issues out of the loop. I measure the execution time. If this is as slow as described by the customer, the SQL statement plus DB seem to be the problem. If not probably something else is the problem (network, application logic …)

I establish a target performance, we need to meet the customer requirements.

I check the execution plan of the statement comparing it with the structure of the schema (tables, indexes, number of rows in tables compared to number of rows actually selected). Are there any obvious problems? For example;

• Full Table Scans that result only in a small percentage of the rows of that table
• Index accesses on a large percentage of rows
• Indexes that don’t get used, although one would expect them to
• Identical expensive pieces of the execution plan, on different places  of the execution plan
• Estimated number of rows in the execution plan that differ significantly from the actual number of rows returned

Depending on what I find I would try some of the following actions, to improve performance

• gather statistics
• create an additional index supporting the where clauses used in the statement
• add columns to an index, in order to remove the need for additional table accesses
• use hints to force certain execution plans
• exchange (not) in with (not) exists and vice versa
• rearrange the select statement (e.g. turn a union of  joins into a join of union)
• partitioning of tables and indexes
• use materialized views
• in very special cases use bitmap indexes or dimensions

With these options the performance of most of the ’slow’ statements I encountered so far could get brought to acceptable levels. Often improving performance by multiple orders of magnitude. When I am able to identify the problem, but not the solution Ask Tom is what I would do next.

There is one answer I get very often, which I consider pretty bad, if it comes without further comment:

Replace outer joins with inner joins

Outer joins have a different meaning than inner joins. Normally you can’t replace one for the other. If you actually can, I’d consider using an outer join a bug, so replacing it with an inner join is bug fixing, not tuning.

Design Patterns. Elements of Reusable Object-Oriented Software: This is the standard book on patterns and patterns are an extremely important piece of vocabulary for every developer. Not to impress coworkers with your knowledge, but to be able to think about bigger concepts than loops and conditionals. It describes each pattern with its purpose, it’s applicability and an example. Make sure to read a couple of blog posts about each pattern, to make sure you avoid some of the more common mistakes when applying them.

Clean Code: A Handbook of Agile Software Craftsmanship: You want to be proud of your code? You want to be able to go back to your code in a year without swearing? This is the book for you. The author Robert C. Martin (also known as Uncle Bob) is extremely aggressive on what he is accepting as clean. This book really changed the way I look at code, and of course the way I write code. Make sure to really understand the lengthy examples. It is tedious, but it pays.

The Pragmatic Programmer. From Journeyman to Master: In Software Development the professionals often follow practices which the professionals them selfs aren’t aware of. This book lists and explains many of these practices as well as some obvious once. While this sounds trivial the resulting book is a great source of small and big practices to acquire as a habit.

For the German readers of my blog: Read the english originals. German translations of computer books are horrible on average.

Agile Software Development with Scrum by Ken Schwaber: Obviously this book is about Scrum, which is just one of many agile processes. But even if you don’t want to go with Scrum, this book will teach you a lot about why Scrum is designed the way it is, and this will be very helpful, no matter what approach you choose. Be warned: the book is small, has crappy graphics and is rather expensive. But it is worth it’s money.

Lean Software Development: An Agile Toolkit by Mary and Tom Poppendieck: This book isn’t tied to any specific process or approach, instead it describes the underlying concepts, which are common to all agile approaches. And these are the ones that really count. Since the Poppendieck have a lot of experience in non software projects they provide lots of examples an cross references which might help convince a stubborn manager, who doesn’t realizes that doing waterfall software development might be 10years behind standards in software development, but is about 50 years behind when compared with other branches of industry.

Management 3.0: Leading Agile Developers, Developing Agile Leaders by Jurgen Appelo: Unfortunately you can’t buy the book yet. But I had the honor to review an early version of the book and provide Jurgen with my nit-picky comments. This book is very different from the other two, because it focuses completely on the management side of things. Agile doesn’t mean managers will go away, they don’t even become useless. Quite the different, they might actually put themselves to good use, and this books will tell you (or your boss) how. Even if you don’t go fully agile, this book will offer lots of advice and ideas, on how to improve you work as a manager.

What are the signs to look out for, that Java is turning into just another kind of COBOL?

What are you going to do when these signs show up?

For many the answer to the first question obviously will be: Java already is the new COBOL. It’s true in the sense that there exists tons of legacy codes, which will continue to exist for the decades to come. But Java has still a big difference: Tons of new projects get started. This includes commercial products as well as FOSS.

Another difference compared to COBOL is: Java is moving forward. Java 7 is (still) announced for 2010, and possibly more significant, new languages are living on the JVM, like Scala, Groovy and Clojure.

This leads to the signs, that mark the point in time to switch to something different:

• No new versions of the JDK. Java never was especially fast or aggressively evolving, but when new versions stop to come at all, or aren’t containing the significant improvements we need, that is a bad sign.
• Companies stop using Java for new projects. Actually when they do stop, it is to late. You should get nervous, when serious discussions are started at large companies that are using mainly Java right now. Obviously this is especially true, when these companies are your customers or employers.
• The Java FOSS community moves to new languages and platforms. Keep an eye on Sourceforge, Apache and Co. are the new and relevant projects still based on and targeted for Java?
• Drop in number of events and number of visitors at events targeted for Java. Does it get empty at your favorite Java conference? Or does it turn into just a couple of booths on a more general targeted conference?

So one day you will notice, that to many of the indicators mentioned above are flashing at you. What do you do?

• Determine if only the language or the Java platform is affected.
• Pick Up a new language or a new platform. Actually when you are early to move, two or three, because you won’t be able to reliable identify the next big thing.
• Do real project using these languages. Get involved in open source projects or do your own in order to really learn the new language.
• Do marketing for your decision. Once you are heading for a new platform, you don’t want to go there alone. There are only few projects that are small enough to be done alone. It is hard to learn, when you are on your own. Get a group going, that can learn and work together. This will at least in the small accelerate the move to the new language. And that is a good thing once you are heading that way.
• Once you are confident to seriously get some work done, using the new tools, push them for your professional projects.

Of course your mileage will vary but for me, it is about time to consider alternatives to Java. That is one of the reasons, why I learned a little Scala, why I will learn groovy and grails, and why I will have a look at Clojure. I’ll do personal projects doing those. And I am marketing that decision right now.

And even if Java stays healthy and Scala dies, I’ll profit from learning new languages and concepts.

• SQL is declarative, which as a side effect means there are now corner cases hidden in if … then … else constructs. In practice this means, that once a SQL statement is working for a couple of cases, it is fairly limited.
• SQL is standard. I worked a lot with Oracle, but when I have to use MySQL, MS SQL Server, DB2 or what ever I’ll be up to speed soon.
• It is pretty powerful. You can do a lot of things with it. Way more than most people thing.
• Well I think that’s about it.

Because on all other accounts I still hate the current state of SQL. And although I am deeply skeptical about all that NoSQL hype lately, I do hope one thing to come out of it: New ideas for a successor of SQL, with a stronger type system, with a powerful scoping a modularization features and a clean design.

Why does SQL suck?

Hard to parse for computers: I don’t know much about parsing, but it seems to be hard for a computer to identify the bits and pieces of a SQL statement, when it is not really SQL but just almost SQL. This results in really crappy IDE support, because the editor doesn’t know what you are trying to write, before you finished it.

Hard to parse for humans: One of the ideas behind the design of SQL was to make it look almost like a sentence. While this works for trivial statements, it completely falls a part when you have some serious statement at hand. For an extremely bad example check out the turing engine implemented in Oracle SQL. Side Note: aren’t some fluent APIs trying to do just that.

Not consistent:Why does the having clause exist? Why nothing similar for group by? Who came up with the syntax of analytic functions. Why can I reference an expression defined the column list in the where clause, but not in the group by clause?

(Almost) no modularization: You need this real complex where condition in another select? Copy it. You need to join the same bunch of table, just with a tiny change in one condition? Copy it. You want to use the same group by clause in a different select? Copy it. There are few little things you can do to break your SQL code, but it is nothing compared to the power of a modern language like … e.g. … Fortran 66.

No decent development environment appart from the target environment When you write a java application (and you do it the way it is supposed to be done), most of it can get compiled, and executed in a simple JVM. No web server needed, nor application server. Compare that to SQL. The only thing that can properly compile and execute a SQL statement is a database. And not just any database, but a database with all the accessed tables created and filled with data. This does not help when writing SQL code.

Ok, we have some reasons for SQL sucking so badly. What’s next. How about asking “Why?” again?

Why are all these things true about SQL?

That answer is easy: I have no idea. So let’s step right ahead and ask again “Why”

Why does a curious developer with over 10 years of experience in database development know some reasons why SQL sucks, but can’t answer the simple question, why these reasons apply?

I think it is because nobody really discusses SQL (at least not loud enough for me to hear). I know there is the No-SQL-Movement, but I happen to think that a relational database is a pretty decent solution to many things. There are also things like the Hibernate Criteria API and HQL. But those do not operate on database schemas, but on classes and objects. While this is fine when you wan’t to work with classes and objects, it sucks when you just want to process large tables. When processiong 100K database records pulling each out of the database, instanciate an object, manipulating it and writing it back is not an option.

Yet a way to write queries in a more modular, easy to parse and easy to test way would be highly appreciated.

The trick is simple to provide data points for every parameter type of the test method. The JUnit Theories Runner will call the test method with every possible combination of datapoints. If you think a little about it you will soon realize some of the limitations of this approach:

• You’ll soon end up with lots of data point fields cluttering your code
• Parameters of the same type will receive the same set of parameters, even when the usable range of inputs is completely different.

Fortunately the developers of JUnit provided really nice solutions to these problems.

Instead of specifying single data points, you can provide a full array of datapoints using the @Datapoints annotation, like so (add imports for good measure):

This of course is much less verbose. Instead of an array you may provide a method returning an array, or at least it looks like this should be possible. But when I tried it JUnit seemed unable to handle the types correctly resulting in IllegalArgumentExceptions. Guess I’ll have to file a bug when finished with this article …

But we still need to take care of parameters which have the same type, but very different meaning and therefore different useful values. The clean OO way of doing things would be to get rid of the generic types like String and use stronger types like CreditCardNumber or Name instead. But then in a perfect world we wouldn’t need tests, because our programs wouldn’t contain any bugs to begin with. So lets try this instead (Again imports omitted):

Wow, thats a lot of code. Just look at the last piece and see what appears in the console when we run it:

just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Amex / Alf tested.
consider Amex / Willie tested.
consider Amex / Tanner tested.
consider Amex / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Master / Alf tested.
consider Master / Willie tested.
consider Master / Tanner tested.
consider Master / Cat tested.
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
consider Visa / Alf tested.
consider Visa / Willie tested.
consider Visa / Tanner tested.
consider Visa / Cat tested.
2
3
4
7
13
23
42

Have a look at the row beginning with: “consider”. Obviously the Theory imagineThisIsATest gets fed with the values from the CreditCardSupplier and NameSupplier. The parameters and the ‘Suppliers’ are connected by the two annotations @AllNames and AllCreditCards. So whenever you have a parameter to a theory where the type alone is not sufficient for identifying the kind of values that should get used, you can simple create an annotation, which itself is annotated with a reference to a ParameterSupplier class and you are all set. You might think this is a lot of code for supplying a handful of parameters. You are right, but remember, that you can reuse your suppliers wherever you need names or credit card values in your tests.

Now let’s look at the first line of the output:
just wanted to show that I can access it @de.schauderhaft.junit.theories.AllNames()
It simply shows of that you get access to the annotation (and actually the signature of the compete test method. This can be very useful, when you want your supplier to behave differently for different theories. Have a look at the NameSupplier above to see how this works.

JUnit actually comes with an example where this is used, and I demonstrated it with the other theory in the demonstration code above. The @TestedOn annotation takes an array of values to be used as data points for the annotated parameter.

Thats it for today. I hope the power of theories became obvious, as well as the power you have as a developer to extend that mechanism. Again be warned: All this nice stuff is in a package named experimental for good reason. If you use it, you might find bugs, and thing will likely change at least in name in an upcoming version. Taking about versions, I am using junit4.8.1 for the examples.

For next week the conclusion of the little series about JUnit theories is planned, with a few thoughts on use and danger of this kind of testing.

When you run this you’ll get this as an output (and one successfull test):

bb a
bb bb
bb ccc
ccc a
ccc bb
ccc ccc

So what is going on? The first thing to note is that this test is executed by a specialized runner of type Theories. This runner executes all the public methods annotated with @Theory. Differently from normal tests theories have parameters. In order to fill these parameters with values, the Theories runner uses all the public fields of matching Type and annotated with @DataPoint. When you take a look at the output, it should be obvious why this is more powerful then parameterized Tests: Every combination of values is tried, thus with e.g. 4 parameters with 4 distinct values each, you end up with 256 test runs.

The idea is to specify a theory about the object under test, that holds for a large class of states and parameters.  Then you provide those as parameters to the test method which will test the theory.

A probably very common case is that a theory is known not to be valid for certain cases. You can exclude these from a test using the Assume class. If an assumption doesn’t hold, the test is silently ignored. This is used in the example above to prevent ‘a’ to be used as a first parameter.

There are some more tweaks to Theories in JUnit, which I will cover in a later blog. Until then enjoy the new feature. But keep in mind the package in which theories reside up to now: org.junit.experimental.theories so I’d expect some changes in the API and at least a change in package name.