Continuous Integration live example: Build your app after every commit

There are a lot of places on the net where you can find a description of what continuous integration is and why it's a good idea.
However, I've always found that the theory only goes so far and that the power of continuous integration (CI) is best expressed thought a live example - and when I say 'live', I mean actually showing you a project running continuous integration and the end products of the process. Let me quickly outline what I will demonstrate in this post:

• Quick feedback - find out the effect of your changes to the codebase in under 10 minutes
• Deployable end product - if your changes are good, you should get a polished end product ready for deployment
  

Here's the story.

Yesterday we'd just finished setting up CruiseControl.Net for my current project. Once done, I proceeded to install CCTray, a little program which sits in your tray and notifies you of the build status. CCTray is pretty important because it ensures that the state of the build is always visible and that the team is always aware of what's going on.

But there was a problem with configuring CCTray. I quickly found that CCTray fails to connect when the Cruise server is behind an SSL connection because of trust issues with the certificate.
Since Owen Rogers (one of the lead developers on CruiseControl.Net) is also on the same project as I am, it took about 30 seconds to figure out where in the code the problem was and a further 10 minutes of Googling to find a fix and apply it.

Usually, it would take few weeks (until the next relaease) for this fix to be available to the public - unless you're one of those brave souls who like to check out the latest code from trunk, build and run. However, this is usually not the path of wisdom with most codebases which don't have tests, since the gods alone know what might have been broken by the last check-in. Which, as a matter of fact, was exactly what I'd done. Owen had written a little NUnit test to replicate the bug, but this didn't actually test anything - indeed we couldn't test this short of setting up a fake Cruise instance behind SSL in the test environment, which is a bit over the top for something this trivial. Anyways, I didn't want to commit this now unnecessary file containing the test because it added no value.

However, I did check in the modified .csproj (Visual Studio project) file which contained a reference to the test (.cs) file by mistake, even though the .cs file wasn't a part of it. I found out five minutes later when the build broke because the compiler was looking for the missing file. So I went back to visual Studio and fixed the .csproj file and checked it in. Another five minutes and we had a green CCNet build.

Most importantly, since the CCNet build runs a battery of unit and integration tests, we can trust the code to run as expected. Of course there will be a few bugs - but far, far less than a codebase without unit and integration tests.

So there you have the first advantage - quick feedback. If someone on your project has committed changes which mess things up, the whole team knows in under five minutes because Cruise goes red and then someone has to go in and fix it (usually whoever broke it in the first place :-)). The idea is to ensure that your codebase is always trustworthy. If it's in the repository, it's good, because you have a Cruise instance constantly integrating it and running tests against it. You're no longer worried about what someone else's commits could have screwed up because Cruise is taking care of checking that.

That way, you always have a clean codebase, which allows you to trust your team members more, so you can update more often and therefore commit more often. Once this cycle is established, you and everyone else on your team will find yourselves checking in every hour or so on average.

For advantage #2 - if you've got an end-to-end build like CC.Net does, then that commit you just made which passed all the tests will end up in a ready-to-deploy bundle created by Cruise. In the case of CC.Net, this is an installer containing the latest code including. Here is the link to the output of the build triggered by the CCTray fix we'd made. As you can see, it is indistinguishable from the releases you'd download from Sourceforge. And this has all the latest fixes!

There are several popular open source projects which make use of CI, including Rails, Rake and Nant. See

http://cruisecontrolrb.thoughtworks.com/projects and
http://ccnetlive.thoughtworks.com/ccnet/server/local/ViewServerReport.aspx

If your CI build pipeline is set up well, you can have a finished, deployable application as the result of every commit. Well, at least those which pass all the tests :-).

Update 2007-07-17:
My apologies for posting my responses to the comments here, but all my attempts to comment on my own blog have been frustrated by the Great Firewall (I'm in China for a month).

Anyways, Siddhi, to answer your question as best as I can - I don't think any free/OSS CI tools (the Cruise family, essentially) support it - yet. You can manage it to some extent by using the 'Project Trigger' which allows one build to watch another even though they're running on different machines. However this cannot be used to fail the initiating build, which is what you asked for. You're basically forced to treat the functional tests running on each different machine/environment as a separate build altogether rather than part of a single one.
Darth's suggestion is also a pretty good idea if you have one big server. That's how we've got things set up on my current projects, with a bunch of VMs and a virtual network.

Zutubi's Pulse, as Jason mentioned, does support this, though. As far as I know, JetBrains' TeamCity doesn't and I'm not sure about Atlassian's Bamboo.