Where is the Science in Software Development?
Lots of stuff in software development has some really solid scientic foundation. Relational databases for example are based on Relational Algebra. There is Lambda Calculus and Type Theory.
And at least in my personal opionion and experience these kind of things work especially well. Yet when it comes to the construction of code the whole thing does look more like a religion. High priest Uncle Bob preaches that shorter is better and that only those developers will reach nirvana that write their tests before the production code.
And than there are the heretics that call all this bull shit, never write a single test but methods as long as the Edda. And lets not forget: those guys ship code as well.
If you check the scientific facts behind any advice of how to write code you are bound to find a void as empty as the space behind Pluto or a reference to some research done 30 years ago with 20 students doing a 4 hour assignement using different styles, languages or what ever. Correlation to my daily work? Approximately zero. Its like the scientific fact that a feather and a hammer should drop with the same acceleration and hence speed. True if you happen live in a vacuum tube, but pretty irrelevant in this form for our daily life.
I’m wondering why is this? It really doesn’t have to be that way. We all use an IDE which already contains lots of plugins. Our builds, including our test runs are automated. The source code is in a version control system. We track our bugs in an issue tracking system, and with a little effort we can link every commit to a bug (or a feature) in our issue tracking system.
Why don’t we have a set of plug ins that gathers various pieces of information about our code development efforts in a central public repository? I imagine such a repository would be enormously valuable for research.
Of course the data wouldn’t be perfect but go talk to biologists, archeologists, economists or any other scientists, with the possible exception of computer scientists, mathmaticians and theoretic physicists: Data is never perfect.
Right now software development is like building cathedrals in mediaval times. Looks impressive as long as it stays up, but nobody can tell if it will stay up without trying.
Its about time we get serious about software development. What is stopping us? Or is this already happening and I just missed it?
There are others agreeing at least in principal with me. And of course there is lots of work done I don’t know about (yet).
If you care at all about software development and evidence, go watch this talk:Greg Wilson – What We Actually Know About Software Development, and Why We Believe It’s True .
Its great, but be warned, it resulted in three more books on my wish list.
Whatever science will bring — it will have a hard time to argue against the religious tendencies.
By the way: True developers cannot reach nirvana, they can only reach /dev/null
The effort that goes under the SEMAT might be somehow related to this topic:
Semat seeks to develop a rigorous, theoretically sound basis for software engineering practice, and its wide adoption by industry and academia.
I like your article very much. It is very helpful for software developers to develop the software application. Thanks for share.
I tend to share some of your doubts about the relevance of many scientific efforts to the daily life of real developers.
That said, there is actually quite a lot of relevant work done in the area of empirically validating the practices of software development (or so-called ‘empirical software engineering’). If you want to dig into this topic further, here are some pointers:
Looks like the links didn’t go through the first time. I’m trying again:
http://www.springerlink.com/content/1382-3256
The primary journal in this area of research
http://esem.cs.lth.se/esem2012/esem/
The primary conference
http://promisedata.org/?cat=11
One of the open data repositories
http://dx.doi.org/10.1109/TSE.2004.43
A paper where 100 professional consultants are used in the experiment
Such a repository already exists. It’s called Uncle Bob, but you chose to dismiss his multi decade long advice.
I’m thinking Uncle Bob is just fed up trying to talk policitally correct about his experience because people will complain anyway. If you just think about it then those heretics you mention never ever back their claims up while Uncle Bob does. So why lump him in the same religious and thus irrelevant category?
Knowing which of the 2 is talking nonsense might be something we should finally figure out.
The biggest problem is that programmers hardly ever refer to computer science papers. The ones that do work in research departments at universities or in research labs at places like IBM, Microsoft and Google.
While in university, no one is encouraged to start researching and to look for journals and articles that will better explain an idea. When at work, employers don’t pay for subscriptions to journals or employees don’t demand subscriptions.
Analysis of project stats such as defect rate and lines of code, etc. has been done, but the research isn’t finding its way into everyone’s hands because we expect any articles referenced to be free or at least very easy to get. We’re very used to the Web and we’ve forgotten or haven’t learned how to do solid research.
Speaking of Greg Wilson’s speech, I just picked up a copy of Making Software and read the first article (chapter?), “The Quest for Convincing Evidence” by Tim Menzies and Forrest Shull. And then I became very sad.
To loosely summarize the chapter, we started out wanting good evidence: elegance, statistical strength, and replicability. But those are hard and no one is really interested in doing that work; coming up with new techniques is easier and more fun. So now, we’re looking for “convincing evidence [that] motivates change” [Yes, their emphasis]. Going forward, we have hundreds of millions of nifty techniques that might or might not have worked for one project, anyway. We have no idea what really works or why, and no real interest in doing the work to find out. What we do have, and in spades, is detailed knowledge of how to convince people to buy into your nifty new idea.
Software development is no different than medicine!!!! Computer science and medical science are both an oxymoron. For medicine to become a science, the discipline needs to be able to handle the combination and permutations of a few billion factors predicting an outcome with 99.99% confidence. This won’t happen in the next few hundred years. Computer science has the same problem. No one can write requirements for a program to cover all of the eventual possibilities that the program will encounter. If they wrote the requirements, probably a 1 billion page document, no programmer could read it in their life time! much less implement it in their life time. Face it. Computer science, computer programming are arts. The difference between the best and the worst is the difference between a 3 year old crayon drawing and Michelangelo – a factor of thousands. Medicine is thousands of years old as a discipline. Computer science is 60 years old! Anyone who thinks that they can turn computer programming into a skilled labor job is clearly ignorant. And the funny thing is that some smart people can’t program a computer because they can’t deal with the failure legs! These are the people who can create a super algorithm and implement it. But if the data falls just a little off of the expectation, the resultant program is garbage. Like medicine it is illogical to try to make computer programming a definitive science. Think about it this way: write a medical diagnostic program. Guess what? The requirements are shaky, imperfect, questionable, wrong, ….. what will the program be?
imho programming is a practice not a science. it depends on taste. it should be compared to other crafts.
Have you even looked into the IEEE Software Engineering Body of Knowledge (SWEBOK)? I would highly recommend it if you think there isn’t a solid set of well-researched and agreed upon data about what works. Software development is always evolving. Languages are changing. Platforms are changing. Chips are changing. All of these changes introduce new possibilities, new room for new techniques. Coming up with general step-by-step principles in an environment constantly in flux is not going to be made any easier by throwing more data at the problem. It’s too big and varied for the Grand Unified Theory approach.
Please take a look at the book Making Software published by O’Reilly. It contains a number of useful research papers on the topic.
You are mistaken in saying that the state of the art for software development practice is like medieval cathedral building. The problem is that software engineering, unlike every other engineering field, has a much wider gulf between best practices and common practice.
Hi everybody,
thanks for the valuable feedback, and sorry for answering so late, somehow notification of new comments didn’t reach my inbox :-/
There are some things comming up in the comments here and over at reddit
I’ll try to answer on some:
1. I never said Uncle Bob is irrelevant or wrong. I’m wearing his green wrist band with pride and agree with most what he says. I also like his not PC way of saying it. What I’m saying is: most of the time he doesn’t even hint an proof of what he says. Same is true for most people, e.g. Fowler and me (which doesn’t mean I want to compare me to Uncle Bob or Martin Fowler)
2. The book Making Software published by O’Reilly
I’m reading it just now. Yes it is a little depressing at places. But it also makes the results of scientific results about software development accessible to non scientists. Very important. A must read.
3. The SWEBOK
I know about it. But I must say I find its use quite limited. Its merely a big dictionary to the research work, that has been done. Its very hard to extract actually valuable information about it. Also I didn’t say that there is no research being done. I tried to make the point, that we could have lots of data to do research with.
4. Software Development is a craft/art, there is no science
While the first part might be correct, the second part is just wrong. Artists and craftman use the results of science all the time. Do you really think masons and architects estimate how thick they have to make a roof? No they have rules, which are based on physics and other sciences.
Just because requirements and programming languages change there is no reason we can’t know if or when pair programming or TDD works.
Bob Martin is a Kool-Aid salesman. Here’s an example of the kind of ideology-laced, evidence-challenged nonsense that you are talking about:
” I want you to believe that Test Driven Development saves time in every case and every situation without exception amen.”
http://blog.8thlight.com/uncle-bob/2012/01/11/Flipping-the-Bit.html
Martin tries to refute a criticism of unit testing by asserting – without any evidence at all – that unit tests always saves time. He also resorts to the dishonest tactic of trying to make the guy he disagrees with defend something that he did not say.
There was an ad-hoc example of the kind of experiment that we need more of several years ago. Here is a summary:
http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html
Some of the links in this article are old but you can still find them.
We need to run “controlled” experiments like this on the various approaches to developing software to see which ones work on which kinds of projects.
Medicine has clinical trials to determine if new drugs and new medical procedures work. Doctors and scientists have developed protocols to assure that the outcomes of these trials are meaningful instead of reflecting an artifact of the trial itself. We will need to develop some reliable protocols for software development experiments so the results are reliable.
Another source we could learn from is the Underwriters Laboratories. They develop tests for all kinds of devices to assure safety and reliability. We need the software development equivalent to the Underwriters Laboratories.
I was just thinking about writing an article with somehow similar title when I stumbled upon yours
Hopefully I’ll have some interesting answers to give … just a little patience for when I’ll find some time to write it.
[...] development. What is stopping us? Or is this already happening and I just missed it? References: Where is the Science in Software Development? from our NCG partner Jens Schauder at the Schauderhaft blog. Posted [...]