On WWRails... finally

I'm not sure why I didn't do this long ago, but better late than never, what?

RubyConf India - Call for Proposals

The Call for Proposals for India's first RubyConf in now on. For information on where and how to submit your proposal, please check the website.

This is a significant milestone to reach when setting up a conference for the first time. There are many people who deserve much appreciation for making this possible, not least all the enthusiastic members of India's Ruby community. A heartfelt thanks to all of you and do come and present; after all it is because of you that the community has gotten this far!

Please do post this information to any Ruby communities that you are a member of. At this point, publicity is key, so please also blog, tweet, digg and reddit away!

RubyConf India - It's official!

Yes, it's true. RubyConf is happening in Bangalore, India early next year. Here's the link to the (currently rather brief) RunConf India 2010 website.

Faster xml deserialisation on JRuby

For the last day or so, I've been looking at XML deserilisation performance at work. When dealing with REST/POX, a respectable fraction of the time is spent in serilising and deserialising xml; consequently it's a fair target for performance analysis. Since I'm currently working on a multi-threaded Twitter library built using Wrest, I decided I might as well take a look at xml deserialisation performance under JRuby.

The Contenders

Wrest delegates XML deserilisation to ActiveSupport, which in turn supports one of three libraries - LibXML Ruby, Nokogiri and of course REXML. REXML is the slowest, buggiest of the three and is pure Ruby. Both LibXML and Nokogiri use the native libxml2 libraries; however LibXML is not available on JRuby whereas Nokogiri is. Nokogiri also has an as yet unreleased version that does not use the JNA based JRuby FFI implementation and is expected to be faster. Build instructions for the non FFI Nokogiri are available here.

REXML can however be enhanced by including JREXML which uses the java xpp3 libs and claims a 10x performance improvemnt.

Thus we have four contenders:

• Vanilla REXML
• REXML enhanced by JREXML
• FFI Nokogiri
• Non FFI nokogiri

The Test Environment

• jruby -v reads jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-05 619cebe) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java])
• I'm using a 2.2GHz 2008 MacBook Pro running Leopard
• The benchmark is a simple Hash.from_xml. It's canned as a Wrest rake task. The full command is jruby -S rake -J-server benchmark:deserialise_xml
• JREXML 0.5.3 and Nokogiri 1.3.3 (the non-FFI build is also 1.3.3, revision fb7e9bb6, from the origin/java branch of tenderlove/nokogiri)

The Numbers

Vanilla REXML

Deserialising using ActiveSupport::XmlMini_REXML
Rehearsal -------------------------------------------------
Hash.from_xml  11.831000   0.000000  11.831000 ( 11.831000)
--------------------------------------- total: 11.831000sec

                    user     system      total        real
hash.from_xml   5.475000   0.000000   5.475000 (  5.474000)

REXML + JREXML

Detected JRuby, JREXML loaded.
Deserialising using ActiveSupport::XmlMini_REXML
Rehearsal -------------------------------------------------
Hash.from_xml  11.323000   0.000000  11.323000 ( 11.323000)
--------------------------------------- total: 11.323000sec

                    user     system      total        real
Hash.from_xml   5.436000   0.000000   5.436000 (  5.436000)

FFI Nokogiri

Deserialising using ActiveSupport::XmlMini_Nokogiri
Rehearsal -------------------------------------------------
Hash.from_xml   9.468000   0.000000   9.468000 (  9.468000)
---------------------------------------- total: 9.468000sec

                    user     system      total        real
Hash.from_xml   3.876000   0.000000   3.876000 (  3.876000)

Non FFI Nokogiri

Deserialising using ActiveSupport::XmlMini_Nokogiri
Rehearsal -------------------------------------------------
Hash.from_xml   5.956000   0.000000   5.956000 (  5.956000)
---------------------------------------- total: 5.956000sec

                    user     system      total        real
Hash.from_xml   2.123000   0.000000   2.123000 (  2.123000)

Conclusion
As expected, REXML was slow. Surprisingly though, JREXML didn't improve those numbers very much.
FFI Nokogiri was faster than REXML, but the JNA seems to have taken its toll - on MRI 1.8.6 the same benchmark runs in under 2s. Non FFI Nokogiri was the real win, though, taking deserialisation performance within spitting distance of the CRuby Nokogiri.

Note that Hash.from_xml does mess around a bit with the hash that the libraries produce and this might make the numbers different from directly using the xml libraries; however since this API is what I need to use with in Wrest (and Rails on JRuby, for that matter) these are the relative performance numbers I'm interested in seeing. If you're not using Rails (or Wrest, dare I say?) you may want to re-run this benchmark against the libraries directly without ActiveSupport mediating.

Wrest, REST and Ruby HTTP libraries

I've been using Rails' ActiveResource for over eighteen months to consume the POX based pseudo REST that Rails applications make so easy to produce, and I'm not overwhelmed.

ActiveResource isn't a particularly well written library, nor is it easy to extend, modify and use. It doesn't support certain features that I've come to consider essential for a Rails POX/REST client, like pagination. The only nice things you can say about it are that it works for the most common cases, and that it ships with Rails. But you know what hackers say about sucky tools - 'Don't get mad; roll your own'. So I did.

I started working on Wrest a few months ago with the intention of creating a drop-in replacement for ActiveResource. The section of the Wrest README that talks about Wrest::Resource is actually a record of the features that we've discussed over the dining table that we wished ActiveResource had. However, as I hacked away on Wrest, I slowly came to realise that a good REST client needs to be a good HTTP client first.

I took a look at those that were popular at the time, but they didn't really appeal to my engineering aesthetic. They heavily favoured class/static methods and seemed to be geared toward command line usage rather than as a library. So I started to implement a clean, easy to use and well encapsulated HTTP library first, resulting in Wrest::Core; it is now ready for use and is sufficiently mature that I'm comfortable writing about it and inviting people to take it for a spin. It does need a few more features, but nothing that can't be added with a few hours worth of hacking. Wrest::Resource however, is still a work in progress, but some of its building blocks are already ready for use. You can see some examples here.

Wrest is available for installation through both RubyGems and as a Rails plugin via Git.

To get the gem, all you need to do is (sudo) gem install wrest. If you want it as a Rails plugin, simply do script/plugin install git://github.com/kaiwren/wrest.git.

Wrest runs on Ruby 1.8, 1.9 as well as JRuby.

Here is an example that shows how you can use the Delicious API using Wrest. If you can't see it (it's a github gist that needs js) you can see the original source here. You may also be interested in the Twitter example, which showcases a more complex scenario with ideas and features from Wrest::Resource.