The General Principle of Software Quality

I’m reading Steve McConnell’s excellent book Code Complete at the moment, which is packed full of useful information.  One thing that particularly jumped out at me is the General Principle of Software Quality.  I’ll get straight to the point, although perhaps a bit of ‘X Factor’-style suspense will spice things up a bit:

The General Principle of Software Quality…………..

Is………………………………

Improving Quality Reduces Development Costs.

This simple but counter-intuitive statement has a potentially massive impact on the way you write software.  The basic premise is that in most software development efforts, finding and fixing defects takes up a large proportion of the construction time, and so writing high-quality code with minimal defects from the start, even though it may require a little more effort upfront, is one of the best ways of reducing the total development time.  (Because as we all know, time is money!)

This being Steve McConnell, of course, everything is backed up by hard data and studies, in this case including NASA’a Software Engineering Laboratory, which found:

.. quality assurance was associated with decreased error rate but did not increase overall development cost

and a study at IBM:

Software projects with the lowest levels of defects had the shortest development schedules and the highest development productivity… software defect removal is actually the most expensive and time-consuming form of work for software

If you’re already coding based on this assumption then, as Mark & Lard would say, Carry On!  (Of course, it’s always good to be sure you’re right, and have some data to back you up in an argument).  But I’ve worked at quite a few places where management and developers alike take the opposite approach – as soon as the schedule starts to get a bit tight, quality is the first thing to suffer, with things like coding standards, testing, and reviews getting squeezed or dropped altogether in an effort to hit that deadline.  Unfortunately, this self-destructive behaviour leads to a vicious cycle which is only ever going to cause the project to get into a worse and worse state, getting delivered way later than it needed to (if it ever gets delivered at all).

So, what is quality code and how do we keep it high?  There are two main areas which are involved that tend to go hand-in-hand: the structure of the code itself (which should be readable, understandable, and generally maintainable), and the number of defects it contains.

Maintainable Code

Code structure and maintainability is important because, as many people have pointed out in the past, code is read more often than it’s written.  Even before the maintenance phase, code is often changed during initial development – functionality is added, defects are fixed, code is refactored, streams are merged.  If the code is hard to understand, it’s much more likely that changes will introduce new defects – and this is magnified during maintenance, when a different set of developers are working on the code.  And when you consider that Yourdon reports that developers making a small (1-5 line) code change have a shocking 50% or more likelihood of introducing a new defect, you can see why easily maintainable code is so important.  I won’t go into detail here about what constitutes maintainable code, but things like well-named variables and routines, cohesive classes and routines (with a strong purpose), short routines, and loosely coupled classes all help.  (See Code Complete or any other development book for more details!)

Once you’ve discovered how to write maintainable code, the best tools at your disposal to ensure it’s being done by your development team are to write a set of coding standards, and then hold code reviews to ensure the standards are being met – more on that later.  Of course, people can’t write good code if they don’t know how, so getting developers up to speed in the first place (using training courses, pair programming, code reviews, etc) is also important here.  And tell them that maintainability is the primary goal of their code, not (for example) minimal lines or performance – you might be surprised how well most developers respond to such goals (which is of course covered in the book, with evidence from yet more studies!) 

Defect-Free Code

Nobody plans to put defects in their code, and in a perfect world it wouldn’t be an issue.  Unfortunately, in this world it’s a very real issue and one that has to be taken into account at an early stage:

Studies at the Software Engineering Institute have found that developers insert an average of 1 to 3 defects per hour into their designs and 5 to 8 defects per hour into code (Humphrey 1997)

Other studies have shown error densities of between 25 and 100 defects per 1000 lines of code, depending on project size.  These defects have many causes – incorrect requirements, design problems, coding errors, mis-communications, changes to poorly understood code, typos, and so on.  Although most are caused by developers and appear in the code, defects in the requirements and design are also common and can have a much greater impact, and also have to be dealt with.

So now that we’ve come to accept that defects are a part of life, what can we do about them?  A large helping of unit tests lovingly applied using TDD will presumably do the trick?  Well, it’s certainly a good start, but only part of the answer according to McConnell.  The following table lists some of the activities which can be used to help find and prevent defects (the number shown is the average percentage of defects in a system found by that method):

Method	Defects Found
Informal Design Reviews	35%
Formal Design Inspections	55%
Informal Code Reviews	25%
Formal Code Inspections	60%
Unit Test	30%
Integration Test	35%
Regression Test	25%
System Test	40%

There were a couple of things that surprised me about this.  The first, as I’ve been concentrating so much on TDD and unit testing lately, was how few of the defects were found by unit testing.  Although it’s not as bad as it seems – the numbers for unit testing were actually from 15 to 50%, so done well it can be more effective, and if you have automated unit tests then this will also cover you to some degree for regression testing too, further boosting it’s usefulness.

The second surprising element was how useful code reviews can be, particularly considering how easy to implement and less time-consuming they are compared to full unit testing.  Some people recommend code reviews as one of the first things you should implement in a development team (for example Roy Osherove), and based on this it’s easy to see why.  Of course, there is a little work involved – get some standards/guidelines in place, decide on a good format for the reviews (the more formal the better, apparently), and so on – but it does seem like quite an easy win for quality.  And it works just as well for requirements and design as it does for code!  Plus you get the extra benefits of spread knowledge, developer learning, and shared code ownership.

A Strategy for Quality

The important thing to take from this is that no single technique is enough – you need multiple methods to ensure high software quality.  Don’t forget that the percentages found are cumulative, so a 50% + 50% detection rate would only find you 75% of the defects.  And the different methods also tend to uncover different kinds of defects and quality issues – no unit test is going to point out all the magic numbers being used, for example, and it can be hard to spot an off-by-one error just by looking at the code.  Different people also tend to spot different errors, another plus point for some kind of review.

Conclusion

I think you get the message by now.  The key point to remember is that you always need to aim for high quality code: Plan your quality strategy by using multiple methods, make sure everyone knows what they are and that quality is a primary goal, and don’t stop doing them at the first hint of a slipping deadline!

Check out this article by Steve McConnell, which covers much the same subject, but with much more authority than I could ever muster.

One more time: Improving Quality Reduces Development Costs! Don’t forget to tell your project manager.