Twitter on Ruby and Scala

Ever heard a code-smell in a conversation? Like this guy is giving you gyan about how he's doing x in language y and it's such a pain in the ass, and you're thinking to yourself "x is such a darn stupid idea in the first place," know what I mean?

The reason I bring this up is because this interview with some of the Twitter hackers came up on the ThoughtWorks software dev mailing list today, and it smelt faintly of cowpats so I thought it worth a mention.

Most of the interview is fairly quiet, sensible stuff; you're ambling along going 'Ho, hum, mildly interesting...' and then something fearsomely ignorant bumbles out and gores you someplace delicate. Here's an example that got pointed out:
Alex Payne: I’d definitely want to hammer home what Steve said about typing. As our system has grown, a lot of the logic in our Ruby system sort of replicates a type system, either in our unit tests or as validations on models. I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.
Aargh. Aargh, I say. I would freakin' bust someone who wrote object-oriented code with is_a? or kind_of? in it, and here's this laddie out on the world wide internetworks proudly admitting that his people write that kind of code. nil checks everywhere? The solution to that has been around for ten years now, for crying out loud. Hasn't anyone told them this kind of code is a people problem, not a language problem?

Here's another bit that seemed suspicious:
Alex Payne: I think programmers who’ve never worked with a language with pattern matching before should be prepared to have that change their perceptions about programming. I was talking to a group of mostly Mac programmers, largely Objective-C developers. I was trying to convey to them that once you start working with pattern matching, you’ll never want to use a language without it again. It’s such a common thing that a programmer does every day. I have a collection of stuff. Let me pick certain needles out of this haystack, whether its based on a class or their contents, it’s such a powerful tool. It’s so great.
Needle in a haystack, eh? In my limited fp experience, pattern matching is used in functional languages for both terseness and for polymorphism. What they seem to be describing - using pattern matching as a glorified regexp and an accessory to the violation of encapsulation - seems pretty unnecessary, given that they're using an OO language that supports polymorphism through objects anyways. This kind of stuff was why I put my Scala studies on hold until I learned how to think correctly in functional terms; Scala makes it easy for a novice to write code that is neither good FP nor good OO.

I have this feeling that Twitter is always 'discovering' something which the rest of the world already knows and has used for a long time. What's worse, they won't go look at the tons of work that's out there and learn from that; no, they'll make the same naive mistakes all over again, like they did a couple of years ago with message queues, a story I've heard from a lot of people.

Don't get me wrong here, I'm not dissing everything they've said; they're good chaps and have a fantastic service deserving of respect. I've also written a little Scala myself and have been lurking on the /Lift/ lists for over a year and I agree with them when they say Scala is a nice language. But frankly, a little engineering and attention to code quality might help them solve more problems than switching languages.

17 comments:

Saager Mhatre said...
This comment has been removed by the author.
Saager Mhatre said...

Somehow pattern matching always seems like a glorified switch-case to me; the eyesore of a linguistically concealed if-else is only exacerbated by the fact that the conditionals are based on the Type of the object being matched. Wait, did I just describe a really cute way of writing...

swith(object.class) {
case Foo: ...
case Bar: ...
...
}

... ?

Egads! @-}

Saager Mhatre said...

Hmm... intewesting! An 'is_a?' hater who writes 'is_a?' code...
Uri#https?

Sidu said...

Isn't

foldr f z [] = z
foldr f z (x:xs) = f x (foldr f z xs)

an aesthetically pleasing use of that switch case? When there are objects involved though, I would tend to agree with you.

About the is_a? allegation, see Connection#http; looks familiar?

My Net::Http code is taken from there because I know they have handled more edge cases than I care to tackle now. But I have my eye on it and it will be gone soon. This kind of code is one of the reasons why ActiveResource annoys me and I'm writing Wrest ;)

Saager Mhatre said...

Yikes! foldr!! Haskell!!!
You just went waay beyond my circle of knowledge. Haskell's still on my 'to pick up ' list. But I was talking about Scala-style-case-class-based-pattern-matching.

Oh, and as for the 'is_a?', why even start by repeating their mistakes?

Darpan said...

Pattern matching is not a glorified switch case like mentioned here by Saager Mhatre. It is a way of making sure you cover all cases which without pattern matching is simply not possible. In the example mentioned, are you sure that there isn't a Baz case you should cover?

As for your object oriented suggestion of never using is_a or instanceof (I am more familiar with this). Well, in a system that is large enough it is simply not possible to make it completely object oriented so that you never have to do an instanceof. And not everyone can code or design like some folks from Thoughtworks (or refactor :) ).

For your null pointer discussion, you should google Tony Hoare and his billion dollar mistake on null pointers. Having option types guarantees not having a null through the type system which is not only a big software engineering advantage but also an efficiency improvement.

Saager Mhatre said...

@Darpan, not sure what you're hinting at, but AFAIK you can write 'match'es in Scala that don't cover all the 'case's. (PS- that comment was very Scala specific)

I agree that pattern matching in pure functional languages such as Erlang and Haskell is much better; flipping the paradigm over from imperative to declarative turns cases into a neat way to specialize problem definitions. Though, I'm not sure if either of them ensures (read enforces) that you cover all cases; but that's probably my inexperience/ignorance speaking.

Darpan said...

Functional languages will give you an error you if you miss a case.

My comment was general about pattern matching, not particularly for Scala. I will have to read what Scala provides but I'd be surprised if it doesn't give a warning or error or mandate a default case. Because exhaustiveness checking is one of the points of pattern matching.

Darpan said...

Correction: functional languages will give you a non-exhaustive match warning.

Darpan said...

The keyword sealed will force exhaustiveness checking of the cases, and the scala compiler will give you a warning.

Saager Mhatre said...

@Darpan-

Hmm... interesting. I'd be a whole lot more appreciative if it were a compile error rather than a warning, though; case analysis should be a fundamental feature.

Either way, Scala allows you to mark pattern matching on even sealed classes as @unchecked and apparently Erlang doesn't even give you a warning! (Yikes and Jinkies!!!)

Saager Mhatre said...

Disclaimer:- My rants above are mostly incited by Scala. I think it's a platform that's trying to achieve too much in one package. But that's just a personal opinion.
- I like functional programming and pure FP-style pattern matching as it forces you to define specializations of as an atomic whole. Tremendous improvement in readability.

Sidu said...

> Well, in a system that is large
> enough it is simply not possible to
> make it completely object oriented
> so that you never have to do an
> instanceof.
There are very special cases (the equals() method for one) where it is acceptable. Otherwise instance_of? indicates bad code at that layer or the layer immediately below it. Basically, it indicates a problem with the people writing the code, not with the language itself.

Option types in Scala are basically nice syntactic sugar for null objects, which is good, but it is still a nice wrapper over an existing OO pattern, not something radically different. No Java/C# codebase I've seen in ThoughtWorks has many nil checks, except at library or input boundaries, and in the latter case they are isolated at the boundaries of the system using null objects.

However, whether you use null objects or a Maybe monad depends on the mechanism for polymorphism; scala which claims to support both OO and functional code seems odd, because my (limited) understanding tells me that dispatch on structural types rather than nominative types is where the real value of pattern matching lies; Scala's pattern match on the other hand seems to match nominative types which simply replicates polymorphic OO based method dispatch. Perhaps I'm missing something?

Darpan said...

Dispatch on structural types using pattern matching maybe an application of pattern matching (although I don't see value since it can be achieved with if instanceof {type defn} else, switch case etc.), but I still think that the real value of pattern matching is exhaustiveness checking, which is a tool for code evolution. Consider (code in ML, but is intuitive enough):
datatype expr =
Numeral of int |
Plus of expr * expr |
Times of expr * expr

Think of datatype as a union type (its a sum type in ML in jargon), it can be Numeral, or Plus or Times

and the function:
fun eval (Numeral n) =
Numeral n
| eval (Plus (e1, e2)) =
let
...
end
| eval (Times (e1, e2)) =
let
...
end

If the definition of expr was changed to:
datatype expr =
Numeral of int
| Plus of expr * expr
| Times of expr * expr
| reciprocal of expr

The compiler will give you a warning at compile time because the function eval is incomplete - it doesn't account for exprs of type reciprocal. Hence, you don't accidentally break code that is in another part of the program.

Options types are not syntactic sugar, they have their own semantics. An option type is defined as

datatype 'a option = NONE | SOME of
'a

(also a sum type), and they are not limited to null pointers substitutes. For example they can be used for functions that expect two parameters but one can be optional, in which case the functions evaluates a default value.

The null object pattern seems unusually unwieldy because with it you may have to define tons of NullMyType classes or inherit all your classes from a Null type.

As an aside, it is not possible for options types as a whole to be syntactic sugar for an object oriented pattern because they have existed in ML since the 70s.

Darpan said...

The examples above were shamelessly ripped off from Practical Foundations for Programming Languages by Robert Harper.

Sidu said...

Sure, I understand what you're saying about the exhaustive check, no problem there. In an OO language, preventing this kind of problem lies in the hands of the developer (encapsulated, unit tested blah blah). No arguments from me on the exhaustiveness checking front. I would disagree with your assertion that polymorphic dispatch isn't that important, but that could just be my ignorance talking (see below for a sample illustrating what I mean).

Option types seem to be syntactic sugar *in Scala* I should have said. In a functional language they make a lot of sense (which is why I mentioned the Maybe monad, which is what I'm more familiar with from Haskell). In Scala, the syntactic sugar saves you from the problem that you just described - having a ton of NullThis/NullThat classes, no doubt about that.

My confusion stems from the fact that in Scala seems to be offering pattern matching on nominative types as an alternative to polymorphic OO based invocation, but *both* are available though they are equivalent (in fact I suspect the latter is implemented by using the former).

Let me illustrate with a Haskell sample from the free Haskell guide (and it uses option types :)):

firstElement :: [a] -> Maybe a
firstElement [] = Nothing
firstElement (x:xs) = Just x

I'm not sure about how something equivalent would be implemented in Scala?

Mushtaq said...

Sidu,

This is how you will do it in Scala.

def firstElement[T](l: List[T]) = l match {
case x :: xs => Some(x)
case _ => None
}

Scala pattern matching is interesting because it marries OO with pattern matching. For pattern matching you need to expose the object composition. This in general goes against encapsulation principal of OO. But Scala enables this exposure in controlled ways:

1. Constructor signature is exposed for only those classes marked as 'case classes'
2. Provides unapply method for more control on what you want to expose

The latter is called "Extractors" and is considered fundamental contribution of Scala. I think pattern matching is still a small part of the whole Scala phenomenon which is trying to bring so many academic ideas to the mainstream.