Road trip to Ooty

In the midst of all the other stuff I've been doing, I clean forgot to mention the awesome trip we had a few weeks ago. This year, the TW Away Day was at Ooty, and the resident hell-raiser, the good Mr. Narla suggested we ride there. Sure enough, in typical Narla style, he went ahead and made it happen. Come October, he had Royal Enfield lined up as sponsors for the trip, supplying us with six Thunderbirds. Apparently they consider it a rather long test ride which is a pretty cool point of view. Besides the six sponsored T-Birds, there was one custom T-Bird (the yellow one in the photos) and two Pulsars.

The fact that they were giving us T-Birds and not Electras worried me - I'd spent a couple of years traveling to college on my friend's T-Bird and I knew it was literally a pain in the ass for the guy on the pillion, which has next to no padding and transmits every bump on the road to the pillion rider's backside with loving attention to detail. This is made worse by the fact that the rider feels no bumps worth mentioning, and so cheerfully races over the worst roads without a care in the world.

Anyways, the Away Day was on the 27th and 28th (the last weekend of October). We left a day earlier on the 26th. Quick bullet point summary, and I'll let the pictures do the rest. You can also read Chris' far more detailed account here.

  • Rained off and on all the way to Mysore. Which was cool - I was completely water proofed and really got to use my Cortech jacket for the first time.

  • Post Mysore to Gudulur was slow, with a couple of incidents along the way, including my bike's key falling out somewhere with neither Vishnu (whom I rode with) nor me noticing.

  • After Gudulur, the roads were totally awesome, Indian style. Vishnu was pillion, luckily, or I wouldn't have survived to reach Bandipur. I mean, they're lucky I'm even using the word 'road'.

  • Bandipur and Mudumali were mind blowing, seriously. Clear roads, sunlight slanting down through the trees and a little gentle rain. The occasional elephant watched us go past. Chris has videos of this bit.

  • After that came the steep climb to Ooty, which we took mostly on first, and which my bike's gearbox didn't quite survive. When we reached the top we found we couldn't engage fourth and fifth gears.

  • A quick stop to refuel at Ooty and we headed to the Holiday Inn where the away day was happening.













Ruby blocks gotchas

New to blocks in Ruby? RubyMonk has chapters covering both introductory topics as well as more detailed lessons on blocks. Do try them out!
There's this thing they say about Ruby - everything is an object. It's true, with very few exceptions, one of them being the block. Well guess what, this little gem of an inconsistency came back to bite me when I was trying to do something involving dynamic redefinition of methods.

The context: I recently wrote a little method decorator to help me figure out the execution times of the methods in a class. Nothing complicated - for a given class, alias each method, then redefine it; the new method invokes the original method while measuring the execution time. Here's a pseudocode-ish example to clarify:
define_method method do |*args|
 t = Time.now

 result = self.send(aliased_original_method, *args)

 diff =  Time.now-t
 puts "#{klass}##{method} took #{diff} s" if diff > 0
 return result
end
You may have already noticed that the psedocode above doesn't handle methods which accept blocks - if I tried to decorate Array, then Array#each would fail to execute. My actual solution did handle this, and I'll publish that in another post, maybe others will find it useful.

Anyways, this didn't take long, but once I was done, I was intrigued by the notion of a generic method decorator. It would be pretty cool if I could include my Decorator module into a class, pass it an arbitrary block to do any of those AOP-ish things like logging or, as I said, measuring execution times, and have all the methods decorated by that block. All the decorator block should have to do to execute the original method would be to yield.

So this led me to try to figure out the whole deal with blocks. Simply put, there are two ways to handle blocks as parameters - implicitly and explicitly.

Implicitly passing and invoking blocks

This is the usual way in which blocks are passed to methods. Here's what it looks like:
def foo(*args)
 yield(args.join(' '))
end
foo('Sidu', 'Ponnappa'){|name| puts "Hello #{name}"} # => "Hello Sidu Ponnappa"
*args allows us to handle an arbitrary number of parameters - they're made available inside the method as an array, where we join them and pass them to our block via yield.
The block is passed to the method by enclosing it in curly braces and placing it after the method invocation. Only one block can be passed to a method in this manner.
Most importantly, the block is never bound, and so is not available as an object. It is implicitly invoked by calling yield within the method.

Explicitly passing, binding and invoking blocks

We go this route if we want a handle to the block. Here's a code example - it's similar to the one above, but we bind to the block and then invoke it explicitly.
def foo(*args, &blk)
 blk.call(args.join(' '))
end
foo('Sidu', 'Ponnappa'){|name| puts "Hello #{name}"} # => "Hello Sidu Ponnappa"
The & binds the block to the variable blk making it available as a Proc object.

An even more explicit style involves first binding a variable to the block and then passing it to the method as an argument (as opposed to using & and having Ruby do it automagically). This style is often used when doing functional programming - Reg Braithwaite has a beautiful article covering this style of programming in Ruby.

Anyways, here's the example:
def foo(*args)
 blk = args.delete_at(-1) # We know that the last argument 
                          # is the bound block
 blk.call(args.join(' '))  
end

the_block = lambda {|name| puts "Hello #{name}"}
foo('Sidu', 'Ponnappa', the_block) # => "Hello Sidu Ponnappa" 
As you can see, we bind the block to the_block using the built in Ruby method lambda and pass it as a regular argument. No magic like the previous examples - the block (now a Proc object) is treated like any other object would be. This, to my eyes, is the most consistent way to use blocks (everything should be an object). It has a significant disadvantage, however, as we'll see in the next section.

The difference - implicit invocation is much faster

The reason why there are two approaches is simple - performance. Binding a block takes time, so we try to avoid it by going the implicit invocation route. Let's get a handle on the actual differences in performance, though, by benchmarking the examples above (modified slightly to avoid 100000 'puts'). I've renamed the three different example methods to foo, bar and ooga respectively.
require 'benchmark'

# Implicit
def foo(*args)
 yield(args.join(' '))
end
puts foo('Sidu', 'Ponnappa'){|name| "Hello #{name}"} # => "Hello Sidu Ponnappa"

# Explicitly binds block when passed
def bar(*args, &block)
 block.call(args.join(' '))
end
puts bar('Sidu', 'Ponnappa'){|name| "Hello #{name}"} # => "Hello Sidu Ponnappa"

# Explicitly binds block before passing
def ooga(*args)
 blk = args.delete_at(-1)
 blk.call(args.join(' '))  
end

the_block = lambda {|name| "Hello #{name}"}
puts ooga('Sidu', 'Ponnappa', the_block) # => "Hello Sidu Ponnappa" 

puts "Starting benchmark"

n = 100000

Benchmark.bmbm(10) do |rpt|
 rpt.report("foo") do
  n.times {foo('Sidu', 'Ponnappa'){|name| "Hello #{name}"}}
 end

 rpt.report("bar") do
  n.times {bar('Sidu', 'Ponnappa'){|name| "Hello #{name}"}}
 end

 rpt.report("ooga") do
  n.times {
    the_block = lambda {|name| "Hello #{name}"}
    ooga('Sidu', 'Ponnappa', the_block)
  }
 end
end
Output:
Hello Sidu Ponnappa
Hello Sidu Ponnappa
Hello Sidu Ponnappa

Starting benchmark

Rehearsal ---------------------------------------------
foo 0.781000 0.000000 0.781000 ( 0.782000)
bar 1.406000 0.000000 1.406000 ( 1.406000)
ooga 1.438000 0.016000 1.454000 ( 1.453000)
------------------------------------ total: 3.641000sec

user system total real
foo 0.782000 0.000000 0.782000 ( 0.781000)
bar 1.375000 0.015000 1.390000 ( 1.406000)
ooga 1.453000 0.032000 1.485000 ( 1.485000)


As you can see, bar, which uses an explicit invocation is approximately 75% slower than foo. ooga, where the block is bound right at the beginning and passed as a parameter is the slowest. TANSTAAFL, I guess.

This trick of benchmarking is borrowed from Joel VanderWerf, who posted a similar benchmark involving all permutations of implicit and explicit invocations over at the Ruby forum.

The catch - implicitly invoking a block from within another block does not work

As a direct consequence of this performance benefit, most of the Ruby code I've seen takes the implicit route. Unfortunately, it is not possible to dynamically redefine methods which expect blocks as implicit parameters - not, and have them continue to behave as before. I know that sounds weird, but read on to the example and all shall be made clear. Hah, always wanted to say that. Ahem.

Getting back to the point, if you dynamically define a method using define_method, the method body is passed to it as a block. You cannot pass a block to this dynamically defined method implicitly - at least not that I could find. If there is a way, please let me know - it would help me get a lot of stuff done neatly. In the meanwhile, here's an example demonstrating this inconsistent behaviour.
class SandBox
  def abc(*args)
    yield(*args)
  end

  define_method :xyz do |*args|
   yield(*args)
  end
end

SandBox.new.abc(1,2,3){|*args| p args}  # => [1, 2, 3]
SandBox.new.xyz(4,5,6){|*args| p args}  # => no block given (LocalJumpError)

SandBox.new.method(:abc).call(1,2,3){|*args| p args} # => [1, 2, 3]
SandBox.new.method(:xyz).call(4,5,6){|*args| p args} # => no block given (LocalJumpError)
The calls to abc succeed, but those to xyz throw a LocalJumpError. There seems to be some fundamental difference in the methods created by def and define_method, with the latter being unable to handle implicitly passed blocks. Here's something else which I tried, which didn't work either:
lmbda = lambda{|*args| yield(*args)}
prc = Proc.new{|*args| yield(*args)}

lmbda.call(7, 8, 9){|*args| p args}  # => no block given (LocalJumpError)
prc.call(10,11,12){|*args| p args}  # => no block given (LocalJumpError)
Note that while lambda and Proc.new both bind a block creating a Proc object, lamda causes the bound block to behave more like a method. It also has some differences in the scope available to the bound block. Proc.new is mildly deprecated in favour of lambda.

To Summarise
  • Blocks violate the 'everything is an object' rule in Ruby for performance reasons. They only become objects when bound to a variable.

  • Implicit invocation of a block using yield is much faster than alternatives involving binding the block to a variable.

  • Most Ruby code uses implicit block passing to avoid binding blocks.

  • Blocks cannot themselves accept a block as an implicit parameter (rather, I couldn't find any way to do this - suggestions welcome).

  • If you define a method using define_method, the method body is passed in as a block. This new method cannot itself make use of yield to invoke an unbound block passed to it implicitly.

  • This is inconsistent behaviour, which, if I haven't missed something, kinda sucks.

While searching for a solution to my problem, I came across Paul Cantrell's exhaustive documentation of the different flavours of blocks/closures in Ruby, as well as their little eccentricities. It's well worth a read.

Update 2007-11-27:
As an anonymous commenter pointed out, Ruby 1.9 will indeed fix this inconsistency. The details can be found here.

You may also want to read: Ruby blocks redux: Ruby 1.9.0, Ruby 1.8.6 and JRuby 1.0.3, which was posted after the release of 1.9.0





Looking for help with your Ruby/Rails project? Hire us!




If you liked this post, you could

subscribe to the feed

or simply comment on this post


Hack hack hack

My friend Akshat's start up, Activ, released a 'micro-blogging to blogspot via SMS' feature a couple of days ago. It's in alpha, running off a home internet connection, but I'm having fun playing around with it anyways.

I'm not very comfortable micro-blogging to my primary blog - the gods alone know what I might say in a moment of enthusiasm - so I created another blog, registered it with Activ and started posting from my phone. All was hunky dory. Now I wanted to embed a summary of my moblog posts in my main blog - something like the Twitter widget.

So first, I created a Yahoo Pipes app to read and modify (mangle?) the feed from my moblog. It takes the feed, truncates the content to a specified (configurable) number of characters and replaces the title of the post with the truncated content.

Next I tried to add this feed to my blog using the 'Display RSS' widget available in Blogger. At first it added it just fine, but a few minutes later Blogger started complaining it was an invalid feed for no reason I could find. I then tried another pipe based feed which had worked earlier and that didn't work anymore either. Curiouser and curiouser. Stumped, I spent some time prodding things in my pipes app hoping something would make the output palatable to Blogger, but no luck. Then inspiration struck - I ran the Pipes output into feedburner and sure enough, the feedburner output was kosher - you can see the result in the sidebar under 'Moblog Summary'.

Next I need to figure out if I can translate the time information embedded in the feed into 'a few minutes ago' and 'a few days ago' a la Twitter widget. All advice from Yahoo Pipes experts welcome.

Scala: initial impressions

I've gotten the lift web framework to build thanks to David Pollak's suggestion on the lists that I reduce allocated memory to 1024MB. Since then I've been ambling around, poking at things, trying to get a feel for both Scala and lift. Here are a few first impressions.

No powerful meta-programming capabilities
Biggest bummer - Scala doesn't seem to have any serious meta-programming capabilities that I could find. Nothing like Groovy and definitely nothing like Ruby. However, two approaches to extending the language are given on the website:
The former allows us to do something like var result = x or y which is rather more readable than the Java equivalent, boolean result = x.or(y);
As you've probably guessed, or is a method defined on a user defined class of which both x and y are instances.

The second option allows us to pass a block to a method in a syntactically clean way. Here's an example from the Scala website.
object TargetTest1 extends Application {
def whileLoop(cond: => Boolean)(body: => Unit): Unit =
if (cond) {
body
whileLoop(cond)(body)
}
var i = 10
whileLoop (i > 0) {
println(i)
i -= 1
}
}

What we're interested in is the definition of the whileLoop construct and its usage. Note that (body: => Unit) allows the whileLoop method to be invoked with a parameterless function as the second argument. In the usage, this is the bit that's encased in curly braces right after whileLoop (i > 0). Pretty nifty. If Java supported blocks using curly braces, the method call would look like this: whileLoop((i > 0), {System.out.println(i); i-=1;})

This makes it possible to construct DSLs which are considerably more readable than those written in, say, Java or C#, because we can get rid of most of the comma/semi-colon noise as well as invoke methods in a eminently readable fashion.

There is also a section on the Scala wiki titled 'future:metaprogramming'. At this time, there are eight sub-sections - Definitions, Design Goals, Requirements, Constraints, Proposals, Examples from other languages, Research Papers and Discussion Threads. Only the last three links lead to any content, so I'm figuring that open classes, eval() and such-like can only be expected in the future.

In the course of my research, I found references to a DSL written in Scala here and discussions on achieving meta-programming on lang.scala here and here.

Functioning package repository
Scala has Scala Bazaar (a.k.a sbaz) which seems to function in a manner similar to RubyGems and Perl's CPAN. However, the list of available packages, at 243, is quite small.

Heavy emphasis on Java
Most of the libraries used for development in Scala are Java libraries. I found little written in pure Scala. It seems the two years of JVM-only development has caused some biasing - if the .Net Scala compiler had kept up with the JVM version, ther'd be a lot more pure Scala stuff available. Something I missed sorely was a pure Scala build tool, something like Ruby's rake. lift for example uses Maven. A lot of the thinking in these areas seems to be from primarily Java people - I came across a project to create a build tool in Scala, but it was based on Ant.

That's it from me on Scala thus far - more as it surfaces.

You may also want to read my previous post on why I'm messing around with Scala and lift: Scala, Lift and being cussed

Barcamp Bangalore 5

I'm planning to attend the next Barcamp happening at IIMB on the 17th and 18th of November, 2007. I suspect most of my time is going to be spent at the hacker, ruby and extreme programming collectives with a bit on the side for any startup related collectives.

The Ruby collective has some interesting people showing up, but no clear agenda or talks (which is fine - this is a Barcamp, eh?). Ditto for the Hacker collective. The Extreme Programming collective is being facilitated by Naresh Jain of ASCI - he has several hands-on sessions in the pipeline which promise to be interesting. By hands on, he means you need to get your laptop with a JDK, Eclipse and a few other things so that you can participate and get your hands dirty. Less talking, more hacking. Topics to be covered include Test Driven Development, refactoring, and other XP related stuff.

If you're interested in these or any other collectives, please remember to register here.

IntelliJ does it again - this time for Ruby

I'd been doing my Ruby work in Jetbrains' IntelliJ 6 using the open source IntelliJ Ruby Plugin. Frankly, it wasn't a patch on developing Java or C# on any half-way decent IDE, including Visual Studio, which just barely manages the half-way decent mark. The plugin could do syntax highlighting correctly most of the time - you occasionally had to add round braces to clarify stuff or it would get confused - and it could run tests. But it couldn't run just one single test in a suite. If you hit the shortcut for run in a test file, it would run all tests. How annoying is that? It also had a few helper menus to do stuff like script/generate and things of that sort. That pretty much summed up its features.

Well, today I upgraded to IntelliJ IDEA 7.01. All was well, I continued where I'd left off a few minutes earlier when I was working in IDEA 6. I write some code. I go to a test. I hit 'Run' and look at the output panel expecting to see the results for the ten tests in the test case. I see just one result. Did I run the wrong test case? A quick check and then I realise what's happened - only the test in which the cursor was positioned has run. It dawned on me that maybe Idea 7 had more to offer than I'd expected.

A second later I was trying out my favourite IDEA shortcuts to see how they worked for Ruby. Shift+F6 (Rename object under cursor), Ctrl+Space (intelligent autocomplete) and Ctrl+Alt+V (Introduce variable) seemed to work fine (I didn't test 'rename' very extensively though).

Intelligent Autocomplete



Extract Variable



Alt+F7 (Find usages) worked, sort of, because it went and found an entirely wrong usage and didn't find another I knew existed. Ctrl+Alt+M (Extract Method) unfortunately doesn't work. Yeah, I know, RDT crossed refactoring's rubicon for Ruby early in 2007, but it looks like it'll be a while before it shows up in production IDEs.

Now I can look forward to some of the stuff I've been missing when developing in Ruby. Like expressive (and consequentially longer) method names and renaming classes and methods when I understand their usage better. I've been using IDEA since it was in version 4, and every single release has made me go 'Whoa, that is so neat' when I looked at some new feature. And they've done it again with IDEA 7.

Incidentally, I have looked at the competition. I've tried Aptana (I've been using RadRails for a year) and have looked at Netbeans. They're OK, but in terms of the overall package, I find IntelliJ the better option. Of course, the $500 price tag attached to IDEA is way too expensive for someone earning in Rupees, so yeah, for my personal stuff I still use Eclipse+Aptana, but I don't enjoy it very much. As for TextMate, I'll consider it when I get a MacBook, provided Apple does something to shed the 'unfriendly toward developers' image it's sprouted since the release of OS X Leopard.

Scala, Lift and being cussed

I'm back after three long weeks full of personal stuff mixed with work which had left me no time for sleep, let alone blogging. Now that I have a little more time, I'm figuring on devoting an hour a day to learning Scala. Quoting from the website:
Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages. It is also fully interoperable with Java.

I've spent some time trying out some of the examples in the Scala tutorials and I like what I see. It's a pure OO language with none of the ugliness we see in Java. It supports the functional style of development. It's statically typed, but supports type inference, so for the obvious cases you don't have to declare variable types.

Why Scala? Well, first, it has a neat Actor library which looks very promising and rather Erlangy (I believe studying and understanding concurrency is important). Second is that it targets the Java JVM, and older versions used to also target the .Net CLR and (so I hear) will do so again in the future. It of course allows interop with existing JVM/CLR libraries. And most importantly, it isn't Erlang.

I'm not saying I have anything against Erlang, but since everyone is going on about Erlang and Yaws and Erlyweb and what have you (interesting discussion here), I'm going to be cussed and pick something else to learn.

Incidentally, the other contender in my 'language to learn this year' contest was Io, which some of my colleagues are already quite fond of (it surfaces on the internal ThoughtWorks developer mailing lists every few months). Io is a pure OO language with Actor based concurrency support, but it uses a prototype-based object model similar to that in Javascript. Understanding one new programming style (using Actors) is going to be hard enough for me with throwing that into the mix.

I'm trying to get beyond the 'Hello World!' stage with Scala by using lift, a fairly young web framework which is supposed to integrate the best features of all the existing frameworks including stuff like support for comet. This is pretty much how I got started with Ruby - I started with Rails and learned Ruby syntax and usage by looking at Rails code. Since this has worked once, I'm hoping it'll work again.

The initial stages of learning a new language always annoy me, when I look at some code and can't understand the syntax (Ruby used to drive up the wall), I don't know where to look for a solution if I have a problem and most importantly, my development environment isn't all figured out. I'm running into some of that already because my initial attempts to compile lift have failed - the Scala compiler ran out of heap space. Nothing to be found online on this. A quick look at the maven build scripts (and I'm not familiar with maven either) reveals the following
<configuration>
<maxmemory>1524m</maxmemory>
</configuration>
1524 MB! Yipe! Not that my laptop can't manage that, but still, 1.5 Gigs? Looks like I'll be spending some time on the lift mailing lists.

Let's see how this works out - I'll post updates whenever something interesting surfaces.