At Leeds Sharp the other day, we started doing the F# Koans as a way of of trying out the language, with the aim of expanding our minds with some functional programming. A bit like coder’s LSD, if you will.
The koans are pretty good at introducing you to things, although initially it’s a bit too easy to get through them without really understanding what’s going on.
However, when you get to the stock example that all changes, as you have to actually use some of what you’ve learned to solve a more involved problem. The purpose of this post is to put up my solution in the hope of getting into a bit of a discussion with others who’ve done it, at Leeds Sharp or otherwise. I have no doubt that my implementation could be improved, so any advice would be more than welcome!
module ``about the stock example`` = let stockData = [ "Date,Open,High,Low,Close,Volume,Adj Close"; "2012-03-30,32.40,32.41,32.04,32.26,31749400,32.26"; // ... "2012-02-29,31.89,32.00,31.61,31.74,59323600,31.74"; ] // function to split a comma-separated string into an array let splitOnCommas (dataAsString:string) = dataAsString.Split([|','|]) // function to take the desired parts of the array as a tuple, // including opening-closing difference let createPriceDifferenceTuple (fullList:string) = ( fullList., abs (Double.Parse(fullList.) - Double.Parse(fullList.)) ) let maximumDifferenceTuple = // split strings into arrays, ignoring header List.map splitOnCommas stockData.Tail // map arrays to tuples |> List.map createPriceDifferenceTuple // get maximum tuple based on second element (price difference) |> List.maxBy snd [<Koan>] let YouGotTheAnswerCorrect() = let result = fst maximumDifferenceTuple // get date, first item in tuple AssertEquality "2012-03-13" result
TekPub is a website set up by Rob Conery and James Avery, two very respected named in the world of development, to deliver video content on many development-related subjects. Rob’s one of my development heroes, who previously worked for Microsoft on ASP.Net MVC, gave us the SubSonic data access framework, and has much good content on the web already in video and blog form.
Anyway, TekPub isn’t free, a year’s subscription costs $200 (or you can pay on a per video or monthly basis), but there are several free videos on there which are well worth watching. (I luckily won a year’s free subscription at a NxtGenUG user group meeting, which is what took me there in the first place).
There are two free series of videos, which are both useful. One is based on Mike Gunderloy’s book ‘Coder to Developer’ and basically discussed how to move from a cowboy to a professional developer, making proper use of things like source control and project organisation tools. Not much new for a seasoned developer, but useful for many currently making the journey.
The other free series is called ‘Concepts‘, which (using C#) covers a lot of the things a growing developer needs to learn – IOC Containers, Unit Testing, Interfaces, and even Behaviour Driven Design (BDD).
And if you’re willing to fork out, there are tens if not hundreds of hours of content on subjects from ASP.Net MVC to iPhone development, jQuery, Git, Ruby, and many more.
So instead of turning on the TV tonight, get over to TekPub and learn something useful instead!
One of my favourite ways of keeping up with development is listening to podcasts, which I usually do in my car – a stereo with a USB port comes in very handy here. (Note: Only do this while alone! The dulcet tones of Scot Hanseman might increase your knowledge, but they’re unlikely to impress the chicks..)
Anyway, audio tips and sexism aside, here are all of the podcasts I’ve been listening to and what I think of them, in order of greatness:
Technology: Mainly .Net
Sound Quality: Average
Hosted by four smart and fairly amusing guys, the beauty of this podcast is that you always get to hear all sides of the argument on any given topic. And unlike some of the other podcasts, the topics are nearly always interesting – I rarely skip an episode. They get some top guests too.
Sound Quality: Excellent
Probably the best-known and longest-running podcast out there, .Net Rocks is perhaps the most professionally produced show out there – you could easily mistake it for a radio show. The presenters are funny and knowledgeable, get good guests and always ask pertinent questions (although you don’t get the same level of discussion as with Herding Code).
Technology: Mainly .Net
Sound Quality: Poor – Average
Run by some of the ElegantCode bloggers, this podcast is similar to Herding Code in it’s style, usually with multiple presenters leading to interesting discussions. It seems to have died since David Starr left for the PluralCast (see below), but I’m hoping it might get resurrected.
Technology: Mainly .Net
Sound Quality: Average
Although this only lasted a short while, the few podcasts it created covered some really useful subjects, in good detail. And being Alt.Net (if you’re aware of that movement), there are also some interesting debates and disagreements in there. Well worth a listen.
Sound Quality: Good
Having recently taking over from the Elegant Code Cast, the Pluralcast has made a good start, covering some interesting subjects. Although run by a commercial company (PluralSight Training), the shows don’t feel like they’re trying to sell you anything. The interviews are sometimes a little one-sided, and the shows don’t always flow as well as they might, but definitely one to try out.
Sound Quality: Excellent
Alongside .Net Rocks, HanselMinutes is the other well-produced, long running podcast. Scott is currently a Senior Program Manager at Microsoft, which means he gets some great guests and inside scoops, but sometimes the interviews are again a little one-sided. Scott’s a really smart guy and knowledgeable developer, and I feel bad for only giving a 6, but the subject is often web development (which I’m not really into), and I often skip shows because I’m not that interested in the guest/subject.
Sound Quality: Good
I’ve really only just started listening to this so it’s a little early to give my opinion – but when did that ever stop a reviewer?! So, as you might imagine from the title, it covers general software engineering in all languages, often with clever academic people being interviewed by other clever academic people. Ok, perhaps I undersell it there, it’s not all academic – and it is good to break out of .Net every now and again.
Sound Quality: Average
A show run my a couple of Microsoft evangelists, this has some good content, but the shows are fairly infrequent and I often skip them. The subjects and interviews can be a little Microsoft-heavy.
Sound Quality: Good
Run (although not hosted) by The Pragmatic Programmers, this podcast consists of interviews with the authors of various books on the Prag Pub label. It’s kind of commercial in that respect, but interesting nonetheless.
Sound Quality: Good
Again I feel bad about giving this such a low rating, as it’s not a bad podcast, I just don’t like it personally. I only listened to the first few, but the topics are usually web-based, and were often covered fairly simplistically, and I didn’t really like the presenter. I think I’ll give it another try soon, as everyone else seems to rate it.
So there you go, plenty to keep you occupied on your lonely commute there! Obviously those ratings are just my opinion, so don’t take my advice – go and try them for yourself!
I’m reading Steve McConnell’s excellent book Code Complete at the moment, which is packed full of useful information. One thing that particularly jumped out at me is the General Principle of Software Quality. I’ll get straight to the point, although perhaps a bit of ‘X Factor’-style suspense will spice things up a bit:
The General Principle of Software Quality…………..
Improving Quality Reduces Development Costs.
This simple but counter-intuitive statement has a potentially massive impact on the way you write software. The basic premise is that in most software development efforts, finding and fixing defects takes up a large proportion of the construction time, and so writing high-quality code with minimal defects from the start, even though it may require a little more effort upfront, is one of the best ways of reducing the total development time. (Because as we all know, time is money!)
This being Steve McConnell, of course, everything is backed up by hard data and studies, in this case including NASA’a Software Engineering Laboratory, which found:
.. quality assurance was associated with decreased error rate but did not increase overall development cost
and a study at IBM:
Software projects with the lowest levels of defects had the shortest development schedules and the highest development productivity… software defect removal is actually the most expensive and time-consuming form of work for software
If you’re already coding based on this assumption then, as Mark & Lard would say, Carry On! (Of course, it’s always good to be sure you’re right, and have some data to back you up in an argument). But I’ve worked at quite a few places where management and developers alike take the opposite approach – as soon as the schedule starts to get a bit tight, quality is the first thing to suffer, with things like coding standards, testing, and reviews getting squeezed or dropped altogether in an effort to hit that deadline. Unfortunately, this self-destructive behaviour leads to a vicious cycle which is only ever going to cause the project to get into a worse and worse state, getting delivered way later than it needed to (if it ever gets delivered at all).
So, what is quality code and how do we keep it high? There are two main areas which are involved that tend to go hand-in-hand: the structure of the code itself (which should be readable, understandable, and generally maintainable), and the number of defects it contains.
Code structure and maintainability is important because, as many people have pointed out in the past, code is read more often than it’s written. Even before the maintenance phase, code is often changed during initial development – functionality is added, defects are fixed, code is refactored, streams are merged. If the code is hard to understand, it’s much more likely that changes will introduce new defects – and this is magnified during maintenance, when a different set of developers are working on the code. And when you consider that Yourdon reports that developers making a small (1-5 line) code change have a shocking 50% or more likelihood of introducing a new defect, you can see why easily maintainable code is so important. I won’t go into detail here about what constitutes maintainable code, but things like well-named variables and routines, cohesive classes and routines (with a strong purpose), short routines, and loosely coupled classes all help. (See Code Complete or any other development book for more details!)
Once you’ve discovered how to write maintainable code, the best tools at your disposal to ensure it’s being done by your development team are to write a set of coding standards, and then hold code reviews to ensure the standards are being met – more on that later. Of course, people can’t write good code if they don’t know how, so getting developers up to speed in the first place (using training courses, pair programming, code reviews, etc) is also important here. And tell them that maintainability is the primary goal of their code, not (for example) minimal lines or performance – you might be surprised how well most developers respond to such goals (which is of course covered in the book, with evidence from yet more studies!)
Nobody plans to put defects in their code, and in a perfect world it wouldn’t be an issue. Unfortunately, in this world it’s a very real issue and one that has to be taken into account at an early stage:
Studies at the Software Engineering Institute have found that developers insert an average of 1 to 3 defects per hour into their designs and 5 to 8 defects per hour into code (Humphrey 1997)
Other studies have shown error densities of between 25 and 100 defects per 1000 lines of code, depending on project size. These defects have many causes – incorrect requirements, design problems, coding errors, mis-communications, changes to poorly understood code, typos, and so on. Although most are caused by developers and appear in the code, defects in the requirements and design are also common and can have a much greater impact, and also have to be dealt with.
So now that we’ve come to accept that defects are a part of life, what can we do about them? A large helping of unit tests lovingly applied using TDD will presumably do the trick? Well, it’s certainly a good start, but only part of the answer according to McConnell. The following table lists some of the activities which can be used to help find and prevent defects (the number shown is the average percentage of defects in a system found by that method):
|Informal Design Reviews||35%|
|Formal Design Inspections||55%|
|Informal Code Reviews||25%|
|Formal Code Inspections||60%|
There were a couple of things that surprised me about this. The first, as I’ve been concentrating so much on TDD and unit testing lately, was how few of the defects were found by unit testing. Although it’s not as bad as it seems – the numbers for unit testing were actually from 15 to 50%, so done well it can be more effective, and if you have automated unit tests then this will also cover you to some degree for regression testing too, further boosting it’s usefulness.
The second surprising element was how useful code reviews can be, particularly considering how easy to implement and less time-consuming they are compared to full unit testing. Some people recommend code reviews as one of the first things you should implement in a development team (for example Roy Osherove), and based on this it’s easy to see why. Of course, there is a little work involved – get some standards/guidelines in place, decide on a good format for the reviews (the more formal the better, apparently), and so on – but it does seem like quite an easy win for quality. And it works just as well for requirements and design as it does for code! Plus you get the extra benefits of spread knowledge, developer learning, and shared code ownership.
A Strategy for Quality
The important thing to take from this is that no single technique is enough – you need multiple methods to ensure high software quality. Don’t forget that the percentages found are cumulative, so a 50% + 50% detection rate would only find you 75% of the defects. And the different methods also tend to uncover different kinds of defects and quality issues – no unit test is going to point out all the magic numbers being used, for example, and it can be hard to spot an off-by-one error just by looking at the code. Different people also tend to spot different errors, another plus point for some kind of review.
I think you get the message by now. The key point to remember is that you always need to aim for high quality code: Plan your quality strategy by using multiple methods, make sure everyone knows what they are and that quality is a primary goal, and don’t stop doing them at the first hint of a slipping deadline!
Check out this article by Steve McConnell, which covers much the same subject, but with much more authority than I could ever muster.
One more time: Improving Quality Reduces Development Costs! Don’t forget to tell your project manager.
Design By Contract comes to .Net
‘Design By Contract’ (DBC) is something the Pragmatic Programmers are very keen on. Introduced as part of the Eiffel language some years ago, it’s a way of specifying how your classes and methods are used in more detail than you get by simple type-checking.
For example, say you had a method that divided some numbers:
The fact that your inputs are Integers means they can only contain certain values, but in this case if ‘Divisor’ contained zero, you’d have a DivideByZeroException on your hands. A contract would enable you to specify that ‘Devisor’ must be positive, or could not be zero. A more common example might be to ensure that an object passed as a parameter was not null.
You can also specify requirements on the return values from methods, or on objects as a whole – ‘whenever this object is in existence, the Foo parameter will not be null’, that kind of thing. By doing this, you’re specifying a contract with users of your code: ‘If you ensure my method is called with the correct parameter values, I’ll guarantee that the return value will be within the specified range’.
This has never been much of a concern to most .Net developers, because although there are some libraries available that implement a kind of DBC for .Net, they are somewhat limited because they’re not part of the language, and certainly not widely used. Generally, people wanting to implement a simple form of this type of checking would do so with exceptions or assertions:
But that’s all about to change! As of .Net 4.0, currently in Beta 2, DBC will be officially part of .Net, under the name Code Contracts. In fact Code Contracts is available now for VS2008 / .Net 3.5 if you want to get started with it straight away. Originally called Spec#, this has been a Microsoft Research project for a while but previously only worked with C#. Now it’s been fully productionized and added into mscorlib, with additional tools for compilation and static checking, and it’s all integrated into Visual Studio.
So now if we wanted to do the above check, we’d need to add a reference to Microsoft.Contracts (if we’re using vs2008), put a using in for System.Diagnostics.Contracts, and add the following code:
‘Wow’, I hear you mutter sarcastically, ‘you’ve replaced an Exception with some slightly different code..’. But there’s more to it than that – for a start, as well as checking parameters, you can specify what a method’s output will be using a contract, and you can also specify that an object of a class will always be in a certain state:
As well as the Contract.Requires() which specifies the valid parameter states, you can see Contract.Ensures() which defines the valid return value of a method, and the method marked with [ContractInvariantMethod] which states that an object of the class must always be in that state. The checks here are pretty basic – you could, for example, check that all of the data items in the return list were valid – but hopefully you get the idea.
So what happens when one of these checks fails? By default, you get an assertion failure, but you can also get a ContractException, or even define your own custom behaviour. You also have the choice of whether the checks should be performed only during development or for the release build. As you can see from the Project Properties screen in Visual Studio, there are a fair few options:
You might have noticed in the screenshot some options for static checking. The static checker is a clever tool which actually analyses your code during compilation and gives you warnings if it can see something that doesn’t meet the specified contracts. For example, if I tried to call my DataGenerator with an invalid parameter of zero:
I would get the following warnings that a contract check was going to fail, including the location of the problem and the contract check:
Good stuff! Although from what I understand, this only works well if you’re very thorough and put contract checks on everything, as validating the code statically like this is a very difficult computing problem.
A few other interesting things that Code Contracts does, which you mainly can’t do with exception-style checking:
- Contracts for interfaces
- Contracts inherited from base classes
- Code Snippets for common situations (e.g. Requires(!String.IsNullOrEmpty())
Aside from the obvious fact that you will find problems with your code more immediately, as Andy Hunt and Dave Thomas point out in The Pragmatic Programmer, the big advantage with doing DBC of any kind is that it makes you think about the valid state of your inputs, outputs and classes. This probably goes as far in improving your code quality as the contract checking itself.
But whether you like the idea or not, as of around March 2010, Code Contracts will officially be a part of .Net, so if I were you I’d start looking into it!
Handy for Unit Tests – and Much More!
I recently found myself having to type some code in repeatedly while unit testing. This sort of thing:
You can see here that I’ve been using Roy Osherove’s naming convention, which I like a lot. I’m also using the AAA (arrange-act-assert) layout for the test code, which I like to comment for increased readability. Although I’ve done something similar many times, I must have seen something about Code Snippets recently as I thought it might be a good idea to give them a try.
Code snippets are a part of Visual Studio (2008 certainly, not sure about before that). They are little code templates that you can get via intellisense, for example when you type ‘if’:
and then hit ‘Tab’ twice, you get the ‘If template’, with the section highlighted ready for you to put your condition in:
As it turns out, Vis Studio doesn’t ship with a snippet editor, but there is a very good open source one on CodePlexwhich is linked to from MSDN. Using this, I can put in my unit test template, and define the ‘tokens’ that I want to replace (in this case, ‘method name’, ‘state under test’ & ‘expected behaviour’):
I’ve also put in the $selected$$end$ bit, which defines where the cursor goes when you’re done. You can see in this one that I’ve put in a #region for the method name as well – the snipped editor’s smart enough to realise this is the same as the one in the method name. Here I’ve defined the shortcut as ‘ter’ (for ‘Test with Region’), because I don’t like typing.
So save that, and Bob’s your uncle! Go back into Visual Studio, type ‘ter’ (tab-tab), and as if by magic, you very own unit test template appears, awaiting your input!
It’s the type of thing you wish you’d found out about years ago, and I’m sure over the next few days I’ll make about a hundred of these things for increasingly pointless bits of code. But it’s another productivity increase, and they all add up, so if I find a few more of these I’ll be able to write a day’s worth of code in about two keystrokes!
Now I wonder what else Visual Studio does that I don’t know about..