Bringing business logic to the browser, or why you should develop in JavaScript

A few weeks ago, I'd observed an interesting side-effect of building an AJAX web application - that the content served asynchronously would not be available to search engines. I'd written a post about it, trying to express the idea that there may be a way of categorising web applications based on whether the content served is the kind that it is valuable to index for search (Wikipedia, Amazon) or where it simply doesn't make sense (Google docs, the vast majority of enterprise web applications). Unfortunately, I made two mistakes. One was that I used the terms 'web site' and 'web application' to distinguish between the two - these terms already mean a lot of things to a lot of people - and second, the idea was raw and my articulation poor.

However, that idea has stayed with me and I've been thinking about it a fair bit since that last post and I've come to the conclusion that what I'd made the focus of my post was merely a side-effect of other, larger trends we're seeing around web frameworks, the relevance of HTML and the way we do web development in general. Please bear with me while I put these ideas down and feel free to flame me to your heart's content in the comments section :-). I'm looking forward to the feedback on this.

First off, I want to get rid of the language baggage for the purposes of this post. I'm going to christen the two types of web applications I have in mind as the web based Information Publisher application and the web based State Interaction application.

An Information Publishing application is a web application whose primary focus is to serve content. The content can be video, audio, text, whatever, but the job of the application is to just serve content. A classic example would be something like Wikipedia or Google search. In both cases, there is information which needs to be served to an audience for consumption. The fact that content in Wikipedia is fairly static while that in Google search is highly dynamic doesn't change the fact that they still essentially publish information.

Contrast this with the web based State Interaction application where the objective of the application is to allow the users to interact with various entities and affect their state (again, Google docs and most enterprise applications). You may publish information as the end product (generate reports, say), but this is not to be confused with the act of creating that information by allowing user interactions with various entities.

The interesting thing is that many web applications have both types and you see one or the other depending on what you're doing with them. Any decent wiki is a good example - when you're in 'edit' mode, it's a State Interaction application (a developer would immediately see entities like Page, Content, History etc.) and when you are viewing wiki content, it's an Information Publisher. The authentication functionality that many websites display is again an example of a State Interaction built into what is otherwise an Information Publisher.

Now, let me explain why I think this distinction is important.
The common underlying infrastructure available to both types of applications is the web browser, with rendering through HTML and communication with the server through gets, posts or AJAX.

In the case of an Information Publisher, this infrastructure is ideally suited to the task. HTML was, after all intended for precisely such uses. Communication with the server to request certain kinds of information can be handled easily and elegantly using name-value pairs (parameters) in a get or post request. A Google search request for the word 'hello' looks like this: http://www.google.co.in/search?q=hello. Nice.
State Interaction applications on the other hand usually have a whole bunch of entities which the user needs to interact with and which usually also need to interact with each other. These live on the server and their interaction with the user is through a user interface rendered using HTML. Changes to the state of an entity or object happen, again, by sending name-value pairs or a json string containing data to the server where these are parsed and some action is taken to alter the states of various objects. To put it bluntly, you have a bunch of objects which demand a high degree of interactivity and we get our UI to talk to them by passing strings around! Not so nice.

The fact that this pain has been felt by developers can be seen in the evolution of web development over time. A bunch of frameworks and tools have been created which build abstractions over this infrastructure to make State Interaction application development easier. For example, ASP.Net 1.1 tried to bring in abstractions which mimicked those used in WinForms. However, attaching an event trigger to a check box (something we do routinely in thick clients) would result in a page reload every time that box was checked or un-checked. The abstraction was defeated by the limitations of infrastructure used (posts). You could develop the same way as you did in WinForms, but the results were far from satisfactory. Things like this made it obvious that while we did need an abstraction, it couldn't really mimic the desktop world where the UI and the model are a method call (or ten :-)) apart. Sure, you're still using a MVC, but very differently from how you would in a thick client. And a whole lot of web frameworks like Spring and Rails have sprung up to support this abstraction. However, these still failed to address the fundamental problem - that highly interactive UI's cannot effectively communicate with their models (data binding, anyone?) by passing strings around.

Why is it that people say thick clients are more interactive than web clients? That this is true is not in doubt, or we wouldn't have such a hullabaloo about AJAX and the responsiveness it introduces. Often, this lack of responsiveness is blamed on the rendering engine of the browser, which renders content using HTML. Obviously, you're told, a markup language cannot be as flexible and easy to develop UI elements in as is a thick client rendering environment using abstractions like Panels, MenusBars and what have you. But this is no longer true since the entire HTML DOM is available for us to manipulate using javascript and the DOM tree structure is remarkably similar to the tree structure of nested widgets in a thick client, something the GWT has used to build up an excellent abstraction - but more on that later in the post.

Therefore, I concluded that this lack of responsiveness has less to do with the rendering medium and is mostly because the communication pipe between the UI and its backing model in a web application is far less effective that that in a thick client.
Which of course begs the question, 'Why have we been so poor in bringing the model from to server the browser?' The obvious answer is performance. Until recently (in fact I'll go so far as to say until the release of Firefox 2) the performance of javascript was so poor as to prevent its utilisation for anything more than a handful of field validations. Creating more than a couple of dozen or so DropAreas on a page using Scriptaculous would make dragging anything so slow that it was next to unusable. But javascript performance has increased in leaps and bounds and it is now possible to actually develop full fledged MVC architectures running purely on the browser, much like a thick client. AJAX is used purely to sync the model on the client with the model on the server, exactly like you would in a thick client. One of the earliest abstractions developed to support this model of State Interaction application design was the GWT. People have shied away from developing applications purely in javascript for many reasons, but the GWT eliminated most of them in one fell swoop. However, there is a general awareness now that developing within a disciplined framework makes life a lot easier (a lesson learned from Rails) and we're seeing javascript MVC frameworks like Jamal and TrimPath surface which are trying to build on this experience to make disciplined development in javascript easier. We've seen pure javascript client applications before in websites like Netvibes, a feed reader with a thick client feel and bunch of desktop UI entities like windows, titlebars and tabs which has been around since 2005. However, it's only now that we're seeing this style of development starting to move into the mainstream.

Of course, these abstractions still stuffer from limitations imposed by the underlying infrastructure. For one, javascript doesn't support threading, so data binding in a GWT application should be handled delicately or you could end up with annoying screen freezes. But all things said and done, these applications are still far more responsive than traditional web applications where the model sits only on the server. As importantly, they are far easier to develop since as I said before, server calls are purely for model syncing and your view objects can talk to model objects using method calls.

Having said that, client side models are a fairly bad idea for a Information Publisher for the simple reason that they're quite unnecessary. The challenge, in my opinion, is to clearly identify which portions of a website require State Interactions and which are good old Information Publishers and implement them accordingly using the appropriate technologies. I'd gotten quite gung-ho about GWT and went and developed an Information Publisher type website using it, only to realise later that none of the content was available through a Google search (I did however fix that issue, but it was a hack which won't scale). You can imagine where this could be crucially important to some websites, especially those which sell products or services. AJAX and dynamic rendering are pretty cool, but should be used appropriately.

To summarise, when developing a web application, it is important to identify which parts of it are information publishers and which are state interactions. As the complexity of the state interactions increases, one should seriously consider bringing the model from the server to the client and using AJAX just to keep the model data in sync. Tools like the GWT, Jamal and TrimPath not only make this possible, but also supply a whole lot of infrastructure (like unit testing and debugging in the GWT, scaffolding in TrimPath) to make the developer's life easier when developing in javascript.

Update: 2007/06/27
I'd originally titled this article 'Why things like the GWT and Jamal are going to help keep web developers sane' but changed it because it was rather vague.

You may also want to read: When should you choose Google's GWT for your web app?

7 comments:

Anonymous said...

Excellent post and a great read. This is a classic client / server scenario. I agree with you on most aspects. The only drawback I see is that in the effort to create 'rich' client side functionality, the first time load / access to a page will yield a huge user delay. This is attributed to the large amount of javascript that needs to be downloaded in order to support the rich client side functionality. Perhaps users would find this one time delay worth their while during use. However, I see this delay having a major impact in cases where users switch between various rich pages frequently. Again, one may argue that all of the web application's functionality could be built into a single page. Or even that all the pages could be opened on disparate browser instances. A lot of web applications however 'prefer' that the application is opened and maintained on a single browser window.

Charles Finkel said...

Hey Sidu, I came across this blog post while digging for opinions on GWT. Then I saw that you work for ThoughtWorks, I company I have been considering applying for for a while, as I feel they align well with my interesting in software development. Please do not feel obligated but I would be very grateful if you can help. My email address is charles.finkel[at]gmail. My homepage is cmanfu.com.

Thanks,
Charles

Jean-Philippe Encausse said...

Hi, really interesting post but I think you don't answer a kind of application mixing both "web based Information Publisher application" and "web based State Interaction application".

I work on Jalios JCMS, a CMS/Portal, so interaction and information, like you explain. But there is not "two world" with sometime one, sometime the other.

Because browsers are faster..., users needs to read/view, publish/update information at the same time.

- If you look at application like WebJam, users edit the portal they saw.
- In modern framework there is "EditInPlaceEditor", ...
- In wiki yes there is BackOffice and FrontOffice but modern software are more "Wysiwyg" mixing Back and Front office.

To summarise,
1. The power of web is flexibility, and ability to mix things easily. Designer might brings cool GUI without development knowledge.

2. GWT brings very good development's ideas but really frozen like desktop application (yon don't add your logo in MS Word Header).

So I think GWT can only be used for "desktop like" application ? But in the futur, web applications will mix more publishing/viewing/editing, in a collaborative way, applications. And GWT will not answer that ?

It would be great in GWT to be able to mix (natively) GWT module with HTML modules in the same HTML web page cleanly.

Anonymous said...

Hi Sidu,

I am a developer based here in Bangalore with over 4yrs of experience. I am interested to be part of TWI. Do you mind if you can help me in forwarding my resume to concerned people.

>> Manjunath (manjunath.d a.t gmail.com)

Anonymous said...

Sidu
It is a shame we cant some how stream the javaascript in behind the page load almost like a ajax back loader that marshals the page load component by component invisably as it were. And shifts priority of the loading threads if a user clicks a component that needs script that hasent loaded at that stage a working sign comes up and that component script gets force loaded and parsed. we perhaps could compress the script as well and decompress it on the client maybe. just random thoughts.

Anonymous said...

i have added further comment here :
http://neildmartin.spaces.live.com/blog/cns!7E48D388265D63E1!406.entry

liz

Shripad Agashe said...

Hi,
I agree with you completely. GWT is my choice of technology if someone asks for RIA in Java. But as mentioned by Husain, the initial load time does play a factor especially in enterprise apps where most of the apps use secure protocol. We cant cache the page and hence every time user access the page, the script would be loaded again.