Continuous Integration live example: Build your app after every commit

There are a lot of places on the net where you can find a description of what continuous integration is and why it's a good idea.
However, I've always found that the theory only goes so far and that the power of continuous integration (CI) is best expressed thought a live example - and when I say 'live', I mean actually showing you a project running continuous integration and the end products of the process. Let me quickly outline what I will demonstrate in this post:
  • Quick feedback - find out the effect of your changes to the codebase in under 10 minutes
  • Deployable end product - if your changes are good, you should get a polished end product ready for deployment
Here's the story.

Yesterday we'd just finished setting up CruiseControl.Net for my current project. Once done, I proceeded to install CCTray, a little program which sits in your tray and notifies you of the build status. CCTray is pretty important because it ensures that the state of the build is always visible and that the team is always aware of what's going on.

But there was a problem with configuring CCTray. I quickly found that CCTray fails to connect when the Cruise server is behind an SSL connection because of trust issues with the certificate.
Since Owen Rogers (one of the lead developers on CruiseControl.Net) is also on the same project as I am, it took about 30 seconds to figure out where in the code the problem was and a further 10 minutes of Googling to find a fix and apply it.

Usually, it would take few weeks (until the next relaease) for this fix to be available to the public - unless you're one of those brave souls who like to check out the latest code from trunk, build and run. However, this is usually not the path of wisdom with most codebases which don't have tests, since the gods alone know what might have been broken by the last check-in. Which, as a matter of fact, was exactly what I'd done. Owen had written a little NUnit test to replicate the bug, but this didn't actually test anything - indeed we couldn't test this short of setting up a fake Cruise instance behind SSL in the test environment, which is a bit over the top for something this trivial. Anyways, I didn't want to commit this now unnecessary file containing the test because it added no value.

However, I did check in the modified .csproj (Visual Studio project) file which contained a reference to the test (.cs) file by mistake, even though the .cs file wasn't a part of it. I found out five minutes later when the build broke because the compiler was looking for the missing file. So I went back to visual Studio and fixed the .csproj file and checked it in. Another five minutes and we had a green CCNet build.

Most importantly, since the CCNet build runs a battery of unit and integration tests, we can trust the code to run as expected. Of course there will be a few bugs - but far, far less than a codebase without unit and integration tests.

So there you have the first advantage - quick feedback. If someone on your project has committed changes which mess things up, the whole team knows in under five minutes because Cruise goes red and then someone has to go in and fix it (usually whoever broke it in the first place :-)). The idea is to ensure that your codebase is always trustworthy. If it's in the repository, it's good, because you have a Cruise instance constantly integrating it and running tests against it. You're no longer worried about what someone else's commits could have screwed up because Cruise is taking care of checking that.

That way, you always have a clean codebase, which allows you to trust your team members more, so you can update more often and therefore commit more often. Once this cycle is established, you and everyone else on your team will find yourselves checking in every hour or so on average.

For advantage #2 - if you've got an end-to-end build like CC.Net does, then that commit you just made which passed all the tests will end up in a ready-to-deploy bundle created by Cruise. In the case of CC.Net, this is an installer containing the latest code including. Here is the link to the output of the build triggered by the CCTray fix we'd made. As you can see, it is indistinguishable from the releases you'd download from Sourceforge. And this has all the latest fixes!

There are several popular open source projects which make use of CI, including Rails, Rake and Nant. See

If your CI build pipeline is set up well, you can have a finished, deployable application as the result of every commit. Well, at least those which pass all the tests :-).

Update 2007-07-17:
My apologies for posting my responses to the comments here, but all my attempts to comment on my own blog have been frustrated by the Great Firewall (I'm in China for a month).

Anyways, Siddhi, to answer your question as best as I can - I don't think any free/OSS CI tools (the Cruise family, essentially) support it - yet. You can manage it to some extent by using the 'Project Trigger' which allows one build to watch another even though they're running on different machines. However this cannot be used to fail the initiating build, which is what you asked for. You're basically forced to treat the functional tests running on each different machine/environment as a separate build altogether rather than part of a single one.
Darth's suggestion is also a pretty good idea if you have one big server. That's how we've got things set up on my current projects, with a bunch of VMs and a virtual network.

Zutubi's Pulse, as Jason mentioned, does support this, though. As far as I know, JetBrains' TeamCity doesn't and I'm not sure about Atlassian's Bamboo.


Anonymous said...

Quick question on CC (ok, its rather long) - Is it possible to have builds run on a different system than the one that CC is running on?

Here is what I'm trying to do. I have a web app. When I check in some code, I want to trigger the build, which in turn runs some selenium tests.

So far no problem.

Now the issue is that I need to run the selenium tests with a variety of browser combinations.

Lets say for example that I need to test with IE6/Win32, IE7/Win32 and Firefox/Linux. Now IE6 and IE7 cannot be installed on the same system, so I have two systems, one with IE6 and one with IE7. Again, Linux needs to be on a third system.

Now when I checkin some code, I need selenium to run the tests through IE6 on computer 1, the same tests through IE7 on computer 2 and again through Firefox on computer 3.

Lets say I have installed selenium on each system along with a build file to run the tests. So I need to execute a build file on each system.

And CC should set the build to green only if all the tests running on the different computers pass.

Can I get CC to do something like this without having to completely hack up the source? Or is there any other CI tool that can handle this?

I figure that this must be a pretty common situation for anyone writing a cross-platform web app. Any ideas?

Chibionos said...

HI...I verified to a recent employee of yours. But I am at a Loss.Can you please tell me how to apply for a posting at thoughtworks. As i know some of the work ethics in the company my ton is jovial. Thought works came to my college sir. But i was unable to attend the first round in time. I really am impressed by the working environment. This has driven me to learn how to apply for the company via Of-campus. Thanks for the time.

Chibionos said...

I came to Write a Comment for Getting info in how to get into Thoughtworks. But this is a bonus...

Siddharta.. I think i have a solution for your problem ..For a Cross platform testing problem like yours. you can do the process with one system. Use a Virtual PC my friend.I was Trying to run my java application on both Linux and Windows.

So i used VM ware... Nice thing for a Virtual PC. And it even gets better.. when you intall the Tools that are a separate download you can actually drag drop files in and out of the VPC[Virtual PC].

But some time that is not working ... :(. But still VM ware is the best solution for your problem.

Or if you are tired of walking from one system to another. Like myself who has three[One main, Downloader, Laptop]. Use Ultra VNC.. I kinda used this to do my Crorepathi Project .. hehehe...

Or i may be completely off from the requirement. If that is the case sorry for your time my friend.

PS: A Dual MON is better for Virtual PC Siddharta my friend..

PS v2.0: If any chance you are sidu.. I am very sorry for the tone of my Blog comment.

Unknown said...


My company's CI server Pulse supports exactly what you are talking about. You can set up build agents on several machines, one for each environment, and can dispatch builds to these machines. You can even set up a build to run across all environments you support in parallel, so the build will only pass if it works in all environments.

Note that Pulse is a commercial product, although free licenses exist for open source projects and small teams.

Anonymous said...

Check out the Hudson CI engine. Highly recommended.

Chibionos said...

Hii its me again... I needed some info on how to apply for you company via of-campus... I heard that there will be a question sent to me and i got to solve it.. I am very anxious as i haven't done a interesting program like what is expected for some time now.. sidu sir.. :).. DO get me the entry level questions..

And a personal query, i dont know what CC mean't sir.. Can you please tell me in a few words what that suppose to abbreviate :) thanks for the time

Ranganath said...

hi, i would like to explore the openings at your company i work on java/j2ee. N i have also applied for the same online on saturday ie 21 july. let me know the chances in your company thank you.