Biweekly Scrummy Scrum

Its inspired by the daily Scrum of a normal Scrum project but happens as you might have guessed every other week. The purpose of this short meeting is to exchange problems and ideas for their resolution and so far it works really well. Although it is really not much time we invest we learn a lot.

It is open to everybody interested in Scrum and it is limited in it time frame. It starts right before the lunch break, so there is a strong incentive for not letting it run to long. So far we don’t use a fixed structure. It is more like an open discussion group. Maybe in the future when we all have somewhat settled for a basic implementation of Scrum we’ll use a more strict structure, where everybody reports about her problems, possible fixes are discussed and a plan is formed about what to do about the problem until the next meeting.

You might think it is a Scrum of Scrums as described in many book, but I don’t think so since the teams involved are completely independent and don’t work on the same or related projects. Also it isn’t only for scrum masters, but for everybody interested. If comparable at all it is more like an Improvement Community as described in Succeeding with Agile, which is a group interested in a certain kind of change, working together in order to support that change. But so far we don’t have an improvement backlog, but maybe that would be a great idea.

What do you to improve you agile/scrum/kanban or projectmanagement skills inside your company?

So when you answered the second question above with ‘yes’ you have a task for tomorrow: Make sure every developer has her own database instance (or schema or whatever you need to work independently from the rest of the team). In addition to that: you need an instance for the automatic tests of your Continuous Integration system, one large one for performance and scalability tests, one for manual tests/integration and of course one for production.

And just in case you are now thinking: “Hell how am I supposed to keep all these databases in sync?” You are having a gapping hole in your software development process. I’ll cover that in a different post.

• I am working on a Swing project (It’s OK, it’s not as bad as it sounds). We have many tests that create components and check certain properties. Although they never get shown in the traditional sense, Swing documentation requires all manipulation of Swing Components to happen in the Event Dispatch Thread. We have an EDTRule, which runs every test using SwingUtilities.invokeAndWait(). It works like a charm. We actually had a dozen tests that would fail under real weird conditions, which behave nicely since they where adorned with an EDTRule. While the Rule takes care on the threading requirements, test stay nicely clean and don’t get poluted by SwingUtilities and Runnable implementations.
• Another class of tests runs scripts against a database. In order to reproduce what the database admins will do, we use SQL/Plus for this (If you don’t know SQL/Plus consider yourself lucky). One problem with these test is, that a small bug in a script might cause SQL/Plus to await user input. In order to prevent the test to hang forever we implemented a TimeoutRule. Each script that gets started puts its Process instance into the rule instance, and the rule instance will destroy the Process after some specified time. Note: JUnit comes with a similar Timeout rule, which will let the test fail after a timeout.
• We have an ApplicationRegistry, which is basically a singleton map containing Implementations for Interfaces. Each test should setup the ApplicationRegistry with whatever is needed, and clean it up afterwards. Often developers forget the last part (When I say developers I am talking about myself). For this we have an ApplicationRegistryRule. It cleans the ApplicationRegistry before the test and after the test. Since it is only one Rule instead of a Before and an After method, it is easier not to forget one half of it. But the Rule does more: Before cleaning it, it checks if the ApplicationRegistry is empty. If it isn’t this means, some other test left it dirty. In this case the Rule calls Assertion.fail(). This way we found (probably) every test which left the ApplicationRegistry dirty. Very cool. A similar approach should work for other central resources like Spring Contexts and the like.

These processes I am talking about look like this:

1. Come up with an idea (I like that process step)
2. Prepare a cost estimate and a business plan for the project
3. Present these to a committee
4. Update the documents from step 2. based on the feedback from step 3. (Its said that many comments are like ‘there should be a company logo on the title page’ or ‘you really shouldn’t center align the page numbers’)
5. Present these to another committee which will decide on the funding of the project.

While this might be of some value of for large projects, it will kill small projects and put medium into trouble right from the beginning. For smaller projects the overhead of the approval process will not only delay the project  and increase the cost by a considerable amount of money. It will also kill of any motivation of anybody in favor of the project.

But it is even worse for large projects. Since once the project ha the approval, who will be inclined to question the project? Of course everybody will be bickering about the stupid idea the project is. But who does step up and says: “You know maybe the guys in control of my salary and career made a mistake and we simply should buy an of the shelve product for 1% of the money?” Correct: No one. The task of questioning the purpose of a project is done by someone else.

This would be OK, if the committee is in a better position to judge these decisions. But it isn’t it has the information of one person, carefully crafted for making sure the decision is made in favor of that person. You might assume just as well, they don’t have any information at all.

How about an approach like this: Give every employee a budget. Top people might get a bigger budget. Employees might invest into a project, or keep the budget. A percentage of the budget left over at the end of the year would get payed the employee. Another percentage would get added to the budget of the next year. When invested into a project, the investing employees get a certain part of the ROI of that project.

Certainly not the best solution. But certainly better then what is going on in companies right now.

If you push towards change,

If you don’t give up,

If you try over and over again,

If you try different approaches,

If you talk to everyone who lets you talk,

someday change will happen no matter what your official title is.

Be persistent if the cause is worth it.

Long Descriptive Name: For example returnsNullWhenCalledOnEmptyList is a nice name for a test

Simple Clean Structure: A test performs necessary setup, executes the method to be tested and performs its assertions. Not more not less

Fast: one thousand tests should run in less than a minute. This allows an extensive test suite to run often during development. If your tests are slower, you might be doing integration tests.

Test a Single Feature or behavior: This is sometimes phrased as ‘one assertion’, which I consider misleading. If you need multiple assertion in order to assert one behavior thats OK, although you might consider a new assert method for that.

Reliable: For the same code base a test should produce the same result every time. A test that fails every 10th time will fail for a real bug and you won’t know it.

Stable: The test only fails, when the code under test doesn’t behave as expected, not because anything else changed.

Easy to Understand: Just as normal code, the intention of a test should be easy to understand. Names help with this. Extracting setup code in a properly named method helps as well.

Should Not Use Production Classes, Except The Class Under Test: This helps with the ‘Stable’ and the ‘Fast’ property. It’s ok to use simple classes like String or Integer.

Don’t Try to Squeeze All Tests for a Class in a Single Test Class: For many classes you will have more then one test class. The different test classes will probably differ in the setup needed for the contained tests.

Independent: No Test should depend on any other Test or on the order of execution.

Let’s start with the well known LazyInitializationException. When you search for this one in the Internet you end up with scenarios like this one:

You load A, close the session you  used and try to access a property of A which gets loaded lazily resulting in a LazyInitializationException. That one is well described, including strategies to avoid it, which basically consist of not closing the session to early.

But there are more hideous scenarios. Assume the classes Mom, Dad and Kid are mapped as one would expect, with references from Mom and Dad to Kid. Now consider the following piece of code:

Depending on your exact Mapping you might get a LazyInitializationException in the last line, although the session is still open. The problem here is that when dad got evicted the kid got evicted as well. And since Mom has a reference to the exact same object, that object isn’t attached to any session anymore. In order to see this effect you’ll have to use mappings like this:

The NonUniqueObjectException gets caused by what could be considered the reverse process. Examine the following piece of code:

The first method call loads a Dad object including Kid from a different session. Then from the ‘main’ session Mom gets loaded including kid. When we attach the Dad object to the session it tries to attach its version of the Kid as well, which collides with the Kid already present in the session.

So how can we avoid these kind of problem? I see several options:

• Don’t use Cascade. Of course this means much more effort on your side in many cases, since you have to carefully track what you need to attach or evict from a session.
• Don’t use evict and don’t reattach existing instances to sessions. I guess this should work in many cases. But I have seen cases where this approach would have caused a large overhead, because reloading complex object graphs.
• Carefully design your cascading boundaries. A look at the Aggregate Root from DDD might help here. An Aggregate Root is a single class which works as a gate keeper (did anybody say facade) to a cluster of classes related to the Aggregate Root. When all the Hibernate related actions go against that Aggregate Root, and cascade from there to the content and only the content of the Aggregate Root you should be safe. All entities outside an Aggregate Root have to use the Aggregate Root or Repositories to gain access. This needs some bookkeeping. But when you do it properly it might actually improve the over all design of your code. Please note that all this is a thought experiment on my side, and I can’t promise that it actually works as intended, but I think it should.

If you want the complete source code for the example, you can download this source.zip file.

Scrum is a Social Thing

Scrum is about team building, Scrum is about the relationship to your customer and the relationship to your users. Let me elaborate, why I think this is the case, and why it is so important for success.

So lets start with some practices of Scrum that are inherently social:

• The complete Team does the estimates. This makes the estimates more reliable, but possibly more important, it is team building. Everybody in the team makes an estimates, everybody commits to the sprint back log, so noone can go “I said all the time it doesn’t work”. We commit as team so if we don’t succeed, it is we that failed, not him.
• The Product Owner is present during estimates. Of course this is important to provide information about the user stories, but it also means she knows all the discussions going on. So if there is a crucial part that everybody forgot about she can’t go: You are professionals, you should have known we need this feature as well. If she didn’t spoke up during planning, she obviously forgot about the important obvious task as well. So everybody is in one boat again.
• Tracking impediments makes sure the impediments get out of the way. But it also communicates, that somebody (the Scrum Master) cares about the problems of the team and works hard to make the environment as productive as possible. This makes a big different, even if some impediments can’t get resolved. Also the team is encouraged to work on the impediments themselves. And trust me, rearranging office furniture together is better then an expensive team building training.
• The Product Owner determines what to achieve in terms of business value, but the team decides how to implement that. This makes a big difference in motivation. If you don’t believe me, get a copy of “Drive” by Daniel H. Pink it will tell you the same.

Hopefully I have convinced you that Scrum contains a lot of social engineering in its practices. But why is this important? Because Agile projects today are done with the same people who did waterfall projects last year. The practices of Scrum don’t make the software development process any better or faster in a direct way. It avoids some waste in the form of irrelevant documentation and features nobody really needs, but when looking at failed projects I don’t think that little avoided waste would make much of a difference.

But team building does. Motivation does. Compare tasks you enjoy and tasks you love, done with a team you hate or fear or a team thats rightly called so. For me an example would be filing my tax compared to creating software. There is easily a factor of ten in productivity … so there is a silver bullet. There is a single thing that might boost performance by a factor of (or close to) ten:

Get the team motivated.

Scrum might help you with that. This also explains why Scrum will be done wrong in many cases: You can’t just apply rules, since rules are a killer for motivation. You have to understand the ideas in scrum, listen to the team and foster the right kind of environment. Then and only then Scrum will increase your chances for success.

Assume you have a fairly complex select statement operating on some serious tables (tens of millions of rows). The customer complains the SQL statement is to slow. What options would you consider for tuning such a select statement? If it matters assume it is an Oracle database.

Just in case you want to apply for a job at a LINEAS here is, what I would consider a damn good answer:

First I check if the SQL statement is really slow, for that I’ll execute the SQL statement with a low level tool (SQL Plus, SQL Developer …) on the database. I fetch the same amount of rows, as the customer (or his application) does. I do that on the DB machine or one a machine with very good connectivity to the database, in order to take network issues out of the loop. I measure the execution time. If this is as slow as described by the customer, the SQL statement plus DB seem to be the problem. If not probably something else is the problem (network, application logic …)

I establish a target performance, we need to meet the customer requirements.

I check the execution plan of the statement comparing it with the structure of the schema (tables, indexes, number of rows in tables compared to number of rows actually selected). Are there any obvious problems? For example;

• Full Table Scans that result only in a small percentage of the rows of that table
• Index accesses on a large percentage of rows
• Indexes that don’t get used, although one would expect them to
• Identical expensive pieces of the execution plan, on different places  of the execution plan
• Estimated number of rows in the execution plan that differ significantly from the actual number of rows returned

Depending on what I find I would try some of the following actions, to improve performance

• gather statistics
• create an additional index supporting the where clauses used in the statement
• add columns to an index, in order to remove the need for additional table accesses
• use hints to force certain execution plans
• exchange (not) in with (not) exists and vice versa
• rearrange the select statement (e.g. turn a union of  joins into a join of union)
• partitioning of tables and indexes
• use materialized views
• in very special cases use bitmap indexes or dimensions

With these options the performance of most of the ‘slow’ statements I encountered so far could get brought to acceptable levels. Often improving performance by multiple orders of magnitude. When I am able to identify the problem, but not the solution Ask Tom is what I would do next.

There is one answer I get very often, which I consider pretty bad, if it comes without further comment:

Replace outer joins with inner joins

Outer joins have a different meaning than inner joins. Normally you can’t replace one for the other. If you actually can, I’d consider using an outer join a bug, so replacing it with an inner join is bug fixing, not tuning.

When her son made a mistake which resulted in loss of his queen and as a result loss of the game she said:

“I better don’t talk to him. I’d only scold him for his mistakes.”

My immediate answer was:

“Of course you should talk to your son! You should praise him for the way he played.”

This resulted in a blank stare.

Is this really so difficult to understand? When somebody makes a mistake, most of the time, they know very well they made a mistake. The chess playing kid knows he lost. He knows losing the queen wasn’t a smart idea. Often kids know this so well that they start crying. Trust me on this: There is no need to tell them what they did wrong. They’ll focus their thoughts on the things they did wrong. And this doesn’t help.

Instead tell them what the did well. A kid playing a game of chess most certainly made a couple of good moves. Your task is to find these moves. Or maybe he remembered to press the clock most of the time after his move. Or he used the available time well to think carefully. Or he remembered to shake hands and setup the board again after the game.

Praise him for that.

It will move the focus to the things he did well. It will turn his focus in something positive. It will motivate him to continue playing and learning. And sooner or later he will start taking care of his queen as well.

And the same applies when you are coaching grown ups. A software developer who introduced a bug knows he made a mistake. And he doesn’t enjoys it. Especially when it was a very damaging bug. chastening him doesn’t help. Instead find the thing he does well. Does a different module of him work well? Did he fix the bug quickly? Does he help his coworkers? Is he especially knowledgeable about a certain area? Did he improve some skills in the past months? Praise him for that.

You might think “But I can’t praise the people that make mistakes! That’s unfair”. You are wrong. You can. You should. You MUST. But of course you should praise the ones doing good work as well. The difference is: Praising the good is much easier and requires much less skill on your side. So if you have problems with praising, you might start with the good ones today.