This is the third part in my little series about dependency anti-patterns. The first part was about the God Node, the second one about the Bypass

Images and nomenclature are based on Degraph.

When you have two slices with lots of parallel connections as in the image on the side, you found this patterns. Just as with real clotheslines there might be some crossing in the lines, but the majority is parallel.

This kind of stuff happens when you split classes into packages (or larger slices) not by the domain they deal with, but by the pattern they implement. An example would be a set of entities and there matching DAOs. If you put all the entities in one package and all the DAOs in another you get exactly this pattern. It is an example of high coupling and low coehesion and in case you are wondering: Yes we normally strive for the opposite.

You want to avoid clotheslines because the affected nodes are couple so tightly together that you can’t really split them, so you might as well throw them in one bag. Which in turn might result in a giant node, you you don’t want either.

The good news is the situation is easy to improve. Just arrange the classes connected by clothesline into one package. Possibly on their own, and if they have cross connections with other pairs, together with those. You’ll end up with something that looks more like the image below: More dependencies inside a package, fewer between packages. Also known as high cohesion, low coupling.

I found the talk intriguing for two reasons. First Kirk looks so much like me that I think I’m seeing me when ever I see a picture of him. It is really unnerving. The other, more important one: He is arguing a similar point as I am.

One of his main messages is: As a responsible developer or an architect you must take care of modules and their dependencies.

While my point is: you should take care of packages and their dependency structure.

So I went and bought his book “Java Application Architecture” in order to learn more about what he has to say.

It turned out to be a book that every advanced developer or architect should read. A label that I haven’t attached to a book for quite some time. Why?

It covers the benefits and costs of a modular software architecture in great detail, something that I haven’t seen in another book. Other books describe how to write code on a very granular level, like methods, classes and patterns. Or they describe the large scale structures relevant in system architecture, but the middle ground of packages and modules gets ignored by most books I have seen so far.

After the general analysis of the effects of modularity, the middle part describes patterns of module dependencies. I personally found this part a little boring and repetitive, because I apply these kind of pattern for quite some time now. But I think it would be very helpful for others who want to apply the tool of modularization to their application.

In the last part Kirk describes how OSGI helps in achieving modularity and reaping the benefits of it. I found this part interesting again, because so far I’ve experienced OSGI only as a road block to productivity and this part together with the rest of the book explained how it actually could have helped me.

Of course the book isn’t perfect either. I think it is to repetitive, which made me skip some passages. Also the title is a little misleading, since it talks only about a very specific part of application architecture. But none of this should keep anybody from buying and reading this book.

As long as the affected nodes are collapsed the Bypass looks just like any circular dependency.

But if you expand them something strange happens. In one node (A) (the main road so to say) one part of the subnodes gets pushed to the left (A1), while the rest gets pushed to the right (A2). The A1 nodes depend on B (the actual bypass) and B depends on A2. Since all arrows still go from left to right it can get hard to spot the problem, when more nodes are involved.

I have seen this many times when someone attempted some kind of dependency injection, but didn’t complete it. Maybe B is a plugin like class implementing a feature for an application A. For this it implements some kind of plugin interface A2, so that A doesn’t actually need to know B. But something (A1) has to plug everything together, so it has to know A1 and B. That is ok, it just shouldn’t be in the same node as A2.

Which already gives a fix to this kind of problem: Move A1 (or A2) to its own node, basically splitting A in two pieces, voila you just got rid of a circular dependency. An alternative is to move A2 into B. The correct decision depends on the semantics of the nodes involved.

I’m using the somewhat abstract term node, which might refer to a class, a package or even to a slice that you define, bundling classes according to some rule that you see fit. If you are confused by the terminology you might have a look at the wiki page for Degraph, where I try to explain these terms.

God Node

When you see a node in a dependency diagram that has tons of in and outgoing connections, you are looking at the death rays emitting from a god node. It does a lot of important things, that’s why it depends on many other nodes. It also gets used by many things, which makes it even worse.

You want to extract stuff from that node! Why? Because when it does so many things and depends on so many things, it is very likely you

1. can’t properly test it
2. have to change it very often
3. thereby over and over creating new bugs, that go undetected by your automatic tests.

If this is a simple node, i.e. it doesn’t represent at least a package or a class with inner classes, may the god class have mercy with your soul. Do some refactoring of that monster, until you can extract some classes. Prefer to extract classes that encapsulate some of the dependencies to other class. This image show how this looks like with the a new class represented by an orange octagon.

An alternative start is to extract interfaces and make other parts of the application depend only on the interfaces. This will move remove some dependencies from the God Node and spread them among the new interface nodes, as sketched in the following image, with the introduced interfaces represented by orange octagons.

Note that

1. Since the god node ‘implements’ the interface nodes it depends on them, not the other way round
2. Some third node has to inject the implementation (the god node) into the clients so it will have dependencies to both. This is basically what I described in an earlier article about how to break dependency cycles.

If this is a compound node have a look inside. Often it already exhibits a structure of it own. Look at the upcoming posts about Dependency Antipatterns for stuff to look out for.

• each and every package belongs to a set of slices, with a slice being either a domain module containing code dealing with a certain part of the domain (like, ‘shopping cart’, ‘fulfillment’, ‘customer’) or a technical part of the application (‘presentation’, ‘persistance’, ‘domain’)
• all dependencies between packages and slices are free of cycles

One problem with this set of rules is: Since dependencies aren’t readily visible in the code it is hard to maintain these rules. And once they are broken it is even harder to fix them.

In one project this lead to developers packing lots of stuff in a single package, because they didn’t know where to put new stuff without creating circles. Obviously that was not what we intended with the rules. But what do you do when faced with a package containing 170 classes and the task to break it (as a first step) in a set of smaller packages void of dependency cycles?

There are tools that tell you when you have dependency cycles. Most of the time they show you the problem as a dependency structure matrix. Unfortunately it is really hard to identify a bunch of classes that you can pull out of a package in order to fix a dependency cycle. They tell you even less about which group of classes you can pull out of a package without creating new dependency cycles.

In order to fix this I created Degraph.

Degraph analyses the dependencies of a bunch of class or jar files and creates a graph representation of it. In this graph it also groups classes into packages and if you want into slices. The result is written to disc as a graphml file. This file can then be opened with yed, a free graph editor with strong layout capabilities.

The result looks like the following image (click for a larger version).

You can see packages as boxes containing classes. If the classes contain inner classes those are contained in the boxes representing the outer classes. What is not visible in the image is that you can open and collapse the boxes representing packages or classes with inner classes. This enables you to show that part of the dependencies that you actually care about.

If this sounds interesting to you give it a try. You can download the current distribution and install it by unzipping it (you’ll need a a properly installed JRE). You can find more instructions on how to use and configure Degraph in its wiki

Degraph is obviously at a very early stage of development, but it is still useful (and fun) if you want to have a look at the structure of your class dependencies. While it shows you the dependency and allows you to identify cycles visually, so far it doesn’t identify such cycles on its own. I’m currently working on adding that ability.

Probably next is the option to integrate Degraph in JUnit (or other) automated tests.

Beyond that I have tons of ideas (definition and test of allowed/disallowed dependencies, creation of dot files, interactive specialized ui, identification of dependencies to break, integration of spring file analysis ..) but this will obviously take some time. In the mean time: If you find a bug, have a problem or an idea for improvement, leave a comment or open an issue.

Updated: Since I released a new version of Degraph I updated the link for downloading the binary distribution.

Line based code coverage  must not fall below 80%.

A method must not be longer than 15 lines of code.

Arguments against such rules state there are cases where such checks limit the freedom of a developer.
During a discussion around these things it occurred to me that the metrics and checks come from a different skill level than the arguments against them.

Let me introduce Shuhari:

Shuhari is a Japanese martial arts concept describing the stages of learning to mastery.

Shu describes the strict adherence to rules as one does when one begins to learn. This is where metrics live. They are strict rules. As a practitioner you just follow the rules without really thinking about them.

Ha describes the bending, changing and breaking of rules, as you might do when you are quite experienced.

Ri finally is described as transcendence, i.e. the rules don’t really have a meaning to a master on this level. She just knows what to do.

I think one can easily see how metrics get in the way of a master at the higher levels. Right? But there is an interesting point in the short Wikipedia article for Shuhari:

Shuhari can be considered as concentric circles, with Shu within Ha, and both Shu and Ha within Ri. The fundamental techniques and knowledge do not change.

For me this means: While a master can and will break rules at will, most of the time she won’t because the rules align with what is the right thing to do ™.

So if you think those code metrics are stupid and shouldn’t apply to your code, you are welcome to prove to me that you are a Ri level master (I’m certainly not). A first indicator would be that you can follow those rules without them actually being enforced.

But if you are constantly breaking builds due to Checkstyle, PMD and Sonar violations, you are not demonstrating mastership but sloppy working.

I learned a lot of stuff during my work on that ship model, but I remember one lesson especially well: How to use file.

You might think using a file can’t be that hard, but trust me you still need quite some advice and practice to do it well.

If you just use it without such practice, you will file of some wood.

The surface will become smooth.

But it won’t become straight.

And you won’t get perpendicular, sharp edges.

You might be able to replace the advice with more practice and maybe some books. But the advice will make progress much faster (and also more frustrating in the beginning because you’ll hear over and over again: “No, you are doing it wrong”).

So my question is who is teaching how to use a file to junior software developers? “File” doesn’t stand in for things like the usage of design patterns or practices like Baby Steps. I’m talking about very basic things that many people consider easy:

• Always start work with a fresh and clean checkout from version control.
• What to do before checking in your code.
• What to write into a commit comment.
• How to write a bug report. 
• …

Most developers learn this kind of stuff at some point in their career, but we as an industry waste a lot of time be forcing them to learn that really slowly. Most teams I have seen do discuss difficult stuff, like design questions and how to use version control to best support the teams work flow. But often there is very little done for teaching the very junior developers.

Often they only get a quick introduction into a tool and then they are left alone with their task.  They work along, do some stuff and only hours or days later they might get some feedback. That feedback often is unspecific and not very helpful like: “Who wrote this stupid commit comment?”

Basically this is a workflow that looks like this:

give task to junior that he might master -> junior works on task -> time goes by -> if lucky junior gets feedback -> repeat with possible adjustment of task difficulty

This might be comfortable for the junior, because he doesn’t get criticized very often, but it slows down learning and this is something we just cannot afford in IT. It also leaves traces of the failures in the results.

I therefore propose to a different approach:

give the simplest task to the junior that you can think of  that he hasn’t proven yet to perform well -> watch him doing it -> stopping him whenever he does something you don’t want to end up in the end result -> explain what is wrong, why it is wrong and how to do it better -> let him try again.

The obvious way of doing this is pair programming, but pair programming for this purpose has some problems. When two developers of very different skill levels work together the senior developer easily ends up doing the work while the junior is watching her. Not helpful. This kind of stuff is about learning, not so much about producing results.

Also with pair programming you have one senior training one junior. While this is awesome for the junior, it is also very expensive. With a little practice on the side of the senior it should be possible to supervise easily six juniors, maybe more.

This again might sound like the typical workshop or tutorial and it is indeed similar. But a typical workshop often stays on the surface of things, because it tries to cover as much as possible. What I propose is an intense training that goes slowly, but strives for perfection.

What do you think? Does this make sense? Is it already happening somewhere?

in most circumstances the unrecognized pitfalls outweigh the benefits

I strongly disagree! But lets go through his points one by one:

Some team members equate coding standards to a lack of confidence in his/her abilities. Others find standards such as spacing and/or comments to be micro-managing.

Pardon me? If you have team members that have an issue adhering to simple team rules like “we want our {s on a separate line” and consider that a personal insult you have some serious issue in your team. Software development is a team effort. If you are working in a team you have to adhere to the rules of the team, no matter if they are written down or exist as implicit knowledge of the team.

Coding standards stifle and discourage coding creativity. Some of the best code comes from an unfiltered mind.
They create unneeded stress within a team as members disagree with various standards. These discussions can spiral into negative, unproductive debates.

Hands up: Who considers it an important part of their creativity to decide on every line how far to indent it? That’s what I thought. Discussions about coding standards? Yup, we have those too. But there is no spiral involved. Somebody brings up a concern like “I think we should stop prefixing field access with this.” We exchange arguments and opinions. Then we decide. End of debate. Again, if you have constant fights in your team about this kind of thing, you have a serious problem in your team and its not the code style guide.

Each new rule adds to the overall complexity of implementation, enforcement, and maintenance of coding standards.

99% of all coding standards get implemented and maintained by your code formatter. The stuff that doesn’t gets enforced by tools like checkstyle. Therefor there really isn’t any relevant effort involved. Coding standards reduce these efforts because it prevents formatting wars and reduces the time needed for reading code.

Each concession to a rule devalues the overall purpose of coding standards.

That’s like saying we should legalize murder since every killing devalues the laws against it. Doesn’t make any sense.

Coding standards distract from the larger goal of building, testing, and releasing functionality in a timely manner.

I still don’t get it. How is it less distracting, when I have to check if a token references a class or a field just by checking if it starts with an upper case character? How is it less distracting, when I have to search for matching curly braces because the code is not properly indented?

Deciding how to handle existing code that pre-dates the creation of a standard is a winless battle.

Again, why battle? Have a discussion, make a decision, be done with it. Get used to it, this is an important part of working in a team.

When a coding standard is not followed, what happens to that code? Must it be fixed immediately? How does that effect its testing, retesting, or release time lines? Some consider this code to be technical debt. That is a difficult label to apply to code if the violations are simply style related.

Since coding standards should be trivial to follow they should break your build and get fixed. If you or team members have a problem with that: Have a discussion, make a decision, be done with it. What is the problem with the label technical debt? Violation of coding standards make it harder to read that code. That is the interest rate you pay for that debt until you pay it back. Fits perfectly. Of course how large that debt is when a single { is misplaced is a different question. But then again, paying that back should be as simple as opening the file and pressing a shortcut.

Proper enforcement of coding standards can consume an unreasonable amount of time.
The decision to enforce coding standards and its consequences must be embraced at all levels within a company. Most companies struggle to embrace doing something twice.

Why is enforcing rules “doing something twice”? Don’t understand that. For the first sentence. Of course you can come up with coding standards that are hard to enforce. Just don’t. Use standards that play well with your tool setting. Automate it. And think again if your coding standard has to be company wide. This works well for small companies, but results in coding standards 1000 pages thick and completely useless. I have seen those. That shouldn’t keep you from having coding standards for your team.

Zac continues to recommend code reviews. A good thing! But not a replacement for coding standards.

You absolutely should have coding standards in any team consisting of two or more developers. Part of deciding on the standards to use should be the way to enforce them and this should be largely automatic.

Even more important than coding standards is a team standard on how to make decisions. If that standard fails in the trivial case of coming up with coding standards you have a serious problem.

And don’t go for compromises when it comes to coding standards (by Geek&Poke)

Sad thing is: While some aspects of package are understood by pretty much everybody in the industry others aren’t. So lets have a look what packages are good for.

Namespaces: By prefixing all your packages with a domain you control, you make sure that your class names are unique. This is essential for the success of an unbelievable number of open source projects. Every project can (and probably does at some stage) define a ‘Filter’ class without having that class interfere with all the other classes of that same name (apart from the poor developer that copied some code without import statements from the web and now has to figure out which Filter class was actually referenced). This one is pretty well understood and I haven’t seen any relevant usage of the root package in ages.

Organization: My son has a huge box of Lego bricks. Probably multiple thousands maybe tens of thousands of them. When he looks for a simple 2×4 brick this is not a problem. But when he is searching for that special brick that only exists 4 times in the collection or even just once!? It might take a loooooong time to find it. Compare that to an apothecary cabinet. Hundreds of drugs and it normally takes only seconds to find the right one. And they don’t even use Google for that! They just have a strict ordering principle where each drug belongs, including a rule how the right box for a new drug is determined. Since everybody involved knows that principle it is easy to determine the correct box where the drug is to be found. Such an ordering principle is tremendously helpful when established early in a project.

When defining such a principle one criteria isn’t sufficient most of the time. But if you use more then one make sure they don’t interfere, by making the rules orthogonal. This means don’t have a rule saying: “All database access code has to go in package x” and another rule stating “All code related to customers has to go into package y”. Otherwise you won’t know where to put the CustomerDAO. Instead apply orthogonal rules on different depths of the package tree. My default package structure looks like this:

<organisational-prefix>.<application>.<deployment-unit>.<module>.<layer>.<optional further substructure if needed>

This results in package names like

com.mycompany.theCoolApp.server.user.persistence

or

com.mycompany.theCoolApp.client.shoppingCart.presentation

If you look at a package structure like this, it becomes pretty obvious where a new class belongs, or where to look if something like this already exists. It gets even better when you avoid names like util or misc which can hide more or less everything. Also you can look at these packages and immediately learn something about the architecture. As soon as you see a level of packages named client, webserver and batchserver you’ll form a model in your head how the application is structured and if the names are picked well it is probably close to the real thing. Since in each module the same rules for layers apply you can find out more about the structure of the application in the lower packages as well.

The modules in between communicate the kind of domain the application deals with. Quite naturally the important concepts get their own package and thereby make a statement to everyone inspecting the code: this is an important concept in this application.

I also like adding the rule “A package should contain a-b classes, but must not contain c or more” with appropriate values for a, b and c. This forces the creation of new packages as the application grows, keeping each package to a manageable size.

Of course on smaller applications the structure might get scaled down. For example if there is just one deployment unit there is no need for a separate package level for that classification.

The last usage for package is the most ignored: the intermediate modeling block: Joe Average Developer concerns herself mostly with classes and methods and single lines of code, while trying to come up with a code structure on that level that fits the needs of the application. Often there is some kind of architect who figures out how to deploy the application and thereby determines the necessary deployment units (think separate jars). If you look at the scale of these artifacts something interesting might catch your eye:

1 method consists of approximately 10 lines of code.

1 class consists of approximately 10 methods.

1 jar consists of approximately 100 – 1000 classes.

If nobody takes care of packages there is at least one, often two levels of structure missing! This gap can and should be filled with packages. This doesn’t only mean the packages should exist and be of reasonable size, it also means they should adhere to common design guide lines. Especially Single Responsibility Principle and proper handling of dependencies:

Single Responsibility Principle With the naming scheme proposed above, a lot of work toward honoring the SRP is done. If the contents of the package does what its name says everything is fine on that front.

Managing of Dependencies is a tougher beast. Java currently doesn’t offer a proper system to control dependencies between packages and especially super packages, i.e. packages that contain multiple other packages. There is OSGI, but I found it a pain in the neck to work with, especially since I never needed all the dynamic loading stuff but suffered from the resulting class loader issues. There is also Jigsaw but this is not there yet. Therefore I prefer homegrown tests for defining and verifying the package structure of applications I work with. My tool of choice is JDepend. It gives you lists of dependencies between packages and you can use those to compare them to rules you define. Somebody creates a dependency from package A to package B that should not exist? Boom, the test turns red.

So what are useful rules for package dependencies? First: No cycles. Not on the package level, but also not on the layer level nor on the module level as used above. Second: Modules and Layers have a strict order in which they can depend on each other, everything else is forbidden.

These rules considerably limit the degrees of freedom one has as a developer. But in my experience it smokes out violations of the Single Responsibility Principle, which often surfaces as cyclic dependencies. For example if you have an Order module and a Customer module it feels like these two need to know each other. If you have an Order, you want to know the Customer it belongs to. If you have a Customer you must be able to tell her the Orders she placed. Right? Yes, probably. But do you need the full blown objects and functionality on both side? Probably not. By coming up with an interface package for example containing only the very core for the customer functionality needed by the Order module and and a separate full Customer module that has the references the orders one can break these dependencies AND achieve a stronger separation of concern in your package structure.

This in turn helps when you try to evolve your application. What today is a package might grow into a deployment-unit someday and if you have circular dependencies between deployment-units you’ll have some serious problems. Or maybe your team grows into multiple teams. With a clean package structure as described above you have a obvious bounds where you can split and also an obvious criteria when the teams have to sit together to discuss changes on a package used by multiple teams.

Take care (of your package structure).

It might look like this:

The User Interface has some representation of the data input by the user, it gets copied over into some kind of transport object (often suffixed TO (Arrrrgh)), which then gets copied over into the domain representation of the data, and finally gets copied over into the persistence representation.

That’s a lot of copying. Not good. In many cases this is just waste. Waste on the runtime side of things when all these objects have to get created and garbage collected and on the implementation side of things when all the stupid code doing the copying is created and maintained. If you look even closer you often see much more copying, when your nice objects get converted to XML, JSON or some other serialized form.

When you see something like this you should start asking questions: copied over?

There are cases where you need to do some copying. For example when you transfer data between layers that are physically separate you have to serialize and deserialize them. But if you want to go from domain object to serialized data stream do you have to go via a transport object? Sure it allows you to use some nice magic library to create your JSON or XML or whatever you want to use. But is it really easier to copy data into a TO and then magically convert it into a JSON structure instead of just creating the JSON structure in the first place, possibly using a little Builder pattern?

If you need different properties in your presentation layer do you have to copy all the stuff over from the domain object? Or maybe you can just provide a thin wrapper providing the additional properties?

I’m perfectly with you if you don’t want the Hibernate mapped entities in your domain logic, because it is bleeding persistence logic in your domain logic. But if you don’t use the Hibernate Entities, is Hibernate really offering a benefit for you?

And beware of reflection for copying. It looks promising in the beginning, but if copying makes any sense at all the structures are different and your reflection based code will become more and more complex and slow. Oh and you might note, that things like Hibernate make extensive use of reflection and/or byte code manipulation.