Twitter on Ruby and Scala

Ever heard a code-smell in a conversation? Like this guy is giving you gyan about how he's doing x in language y and it's such a pain in the ass, and you're thinking to yourself "x is such a darn stupid idea in the first place," know what I mean?

The reason I bring this up is because this interview with some of the Twitter hackers came up on the ThoughtWorks software dev mailing list today, and it smelt faintly of cowpats so I thought it worth a mention.

Most of the interview is fairly quiet, sensible stuff; you're ambling along going 'Ho, hum, mildly interesting...' and then something fearsomely ignorant bumbles out and gores you someplace delicate. Here's an example that got pointed out:

Alex Payne: I’d definitely want to hammer home what Steve said about typing. As our system has grown, a lot of the logic in our Ruby system sort of replicates a type system, either in our unit tests or as validations on models. I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.

Aargh. Aargh, I say. I would freakin' bust someone who wrote object-oriented code with is_a? or kind_of? in it, and here's this laddie out on the world wide internetworks proudly admitting that his people write that kind of code. nil checks everywhere? The solution to that has been around for ten years now, for crying out loud. Hasn't anyone told them this kind of code is a people problem, not a language problem?

Here's another bit that seemed suspicious:

Alex Payne: I think programmers who’ve never worked with a language with pattern matching before should be prepared to have that change their perceptions about programming. I was talking to a group of mostly Mac programmers, largely Objective-C developers. I was trying to convey to them that once you start working with pattern matching, you’ll never want to use a language without it again. It’s such a common thing that a programmer does every day. I have a collection of stuff. Let me pick certain needles out of this haystack, whether its based on a class or their contents, it’s such a powerful tool. It’s so great.

Needle in a haystack, eh? In my limited fp experience, pattern matching is used in functional languages for both terseness and for polymorphism. What they seem to be describing - using pattern matching as a glorified regexp and an accessory to the violation of encapsulation - seems pretty unnecessary, given that they're using an OO language that supports polymorphism through objects anyways. This kind of stuff was why I put my Scala studies on hold until I learned how to think correctly in functional terms; Scala makes it easy for a novice to write code that is neither good FP nor good OO.

I have this feeling that Twitter is always 'discovering' something which the rest of the world already knows and has used for a long time. What's worse, they won't go look at the tons of work that's out there and learn from that; no, they'll make the same naive mistakes all over again, like they did a couple of years ago with message queues, a story I've heard from a lot of people.

Don't get me wrong here, I'm not dissing everything they've said; they're good chaps and have a fantastic service deserving of respect. I've also written a little Scala myself and have been lurking on the /Lift/ lists for over a year and I agree with them when they say Scala is a nice language. But frankly, a little engineering and attention to code quality might help them solve more problems than switching languages.

Scala: initial impressions

I've gotten the lift web framework to build thanks to David Pollak's suggestion on the lists that I reduce allocated memory to 1024MB. Since then I've been ambling around, poking at things, trying to get a feel for both Scala and lift. Here are a few first impressions.

No powerful meta-programming capabilities
Biggest bummer - Scala doesn't seem to have any serious meta-programming capabilities that I could find. Nothing like Groovy and definitely nothing like Ruby. However, two approaches to extending the language are given on the website:

• any method which takes a single argument may be used as an infix or postfix operator
  
• closures are constructed automatically depending on the expected type (target typing)

The former allows us to do something like var result = x or y which is rather more readable than the Java equivalent, boolean result = x.or(y);
As you've probably guessed, or is a method defined on a user defined class of which both x and y are instances.

The second option allows us to pass a block to a method in a syntactically clean way. Here's an example from the Scala website.

object TargetTest1 extends Application {
def whileLoop(cond: => Boolean)(body: => Unit): Unit =
  if (cond) {
    body
    whileLoop(cond)(body)
  }
var i = 10
whileLoop (i > 0) {
  println(i)
  i -= 1
}
}

What we're interested in is the definition of the whileLoop construct and its usage. Note that (body: => Unit) allows the whileLoop method to be invoked with a parameterless function as the second argument. In the usage, this is the bit that's encased in curly braces right after whileLoop (i > 0). Pretty nifty. If Java supported blocks using curly braces, the method call would look like this: whileLoop((i > 0), {System.out.println(i); i-=1;})

This makes it possible to construct DSLs which are considerably more readable than those written in, say, Java or C#, because we can get rid of most of the comma/semi-colon noise as well as invoke methods in a eminently readable fashion.

There is also a section on the Scala wiki titled 'future:metaprogramming'. At this time, there are eight sub-sections - Definitions, Design Goals, Requirements, Constraints, Proposals, Examples from other languages, Research Papers and Discussion Threads. Only the last three links lead to any content, so I'm figuring that open classes, eval() and such-like can only be expected in the future.

In the course of my research, I found references to a DSL written in Scala here and discussions on achieving meta-programming on lang.scala here and here.

Functioning package repository
Scala has Scala Bazaar (a.k.a sbaz) which seems to function in a manner similar to RubyGems and Perl's CPAN. However, the list of available packages, at 243, is quite small.

Heavy emphasis on Java
Most of the libraries used for development in Scala are Java libraries. I found little written in pure Scala. It seems the two years of JVM-only development has caused some biasing - if the .Net Scala compiler had kept up with the JVM version, ther'd be a lot more pure Scala stuff available. Something I missed sorely was a pure Scala build tool, something like Ruby's rake. lift for example uses Maven. A lot of the thinking in these areas seems to be from primarily Java people - I came across a project to create a build tool in Scala, but it was based on Ant.

That's it from me on Scala thus far - more as it surfaces.

You may also want to read my previous post on why I'm messing around with Scala and lift: Scala, Lift and being cussed

Scala, Lift and being cussed

I'm back after three long weeks full of personal stuff mixed with work which had left me no time for sleep, let alone blogging. Now that I have a little more time, I'm figuring on devoting an hour a day to learning Scala. Quoting from the website:

Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages. It is also fully interoperable with Java.

I've spent some time trying out some of the examples in the Scala tutorials and I like what I see. It's a pure OO language with none of the ugliness we see in Java. It supports the functional style of development. It's statically typed, but supports type inference, so for the obvious cases you don't have to declare variable types.

Why Scala? Well, first, it has a neat Actor library which looks very promising and rather Erlangy (I believe studying and understanding concurrency is important). Second is that it targets the Java JVM, and older versions used to also target the .Net CLR and (so I hear) will do so again in the future. It of course allows interop with existing JVM/CLR libraries. And most importantly, it isn't Erlang.

I'm not saying I have anything against Erlang, but since everyone is going on about Erlang and Yaws and Erlyweb and what have you (interesting discussion here), I'm going to be cussed and pick something else to learn.

Incidentally, the other contender in my 'language to learn this year' contest was Io, which some of my colleagues are already quite fond of (it surfaces on the internal ThoughtWorks developer mailing lists every few months). Io is a pure OO language with Actor based concurrency support, but it uses a prototype-based object model similar to that in Javascript. Understanding one new programming style (using Actors) is going to be hard enough for me with throwing that into the mix.

I'm trying to get beyond the 'Hello World!' stage with Scala by using lift, a fairly young web framework which is supposed to integrate the best features of all the existing frameworks including stuff like support for comet. This is pretty much how I got started with Ruby - I started with Rails and learned Ruby syntax and usage by looking at Rails code. Since this has worked once, I'm hoping it'll work again.

The initial stages of learning a new language always annoy me, when I look at some code and can't understand the syntax (Ruby used to drive up the wall), I don't know where to look for a solution if I have a problem and most importantly, my development environment isn't all figured out. I'm running into some of that already because my initial attempts to compile lift have failed - the Scala compiler ran out of heap space. Nothing to be found online on this. A quick look at the maven build scripts (and I'm not familiar with maven either) reveals the following
<configuration>
<maxmemory>1524m</maxmemory>
</configuration>
1524 MB! Yipe! Not that my laptop can't manage that, but still, 1.5 Gigs? Looks like I'll be spending some time on the lift mailing lists.

Let's see how this works out - I'll post updates whenever something interesting surfaces.