September 10, 2011

A New Idea for Front-End Development

I've been doing front-end web development for almost 15 years now, and for that entire time this the typical development model has looked something like this:

  1. Start with a page design and mark up a single page, using the templating language of your choice to dynamically generate content. Create some CSS to make it look the way you want.
  2. Use javascript to handle realtime interactions (hiding/showing stuff, XHR transactions, etc).


As web applications have grown more dynamic on the front end, that second step has grown to encompass most of a developer's time, but we haven't really changed much about the first step. A perfect example is the way jQueryUI works: you create your markup in the html template as usual, but then you add one or more classes to it--for example, "ui-widget" or "ui-corners-all". When the page loads, jQueryUI's init() method crawls the DOM looking for those classes and attaches event bindings, behaviors, extra markup, etc., to them.

In theory this is a great idea, but in practice it's a nightmare of inconsistency; every widget is marked up differently according to the needs at the time of its design. This is because jQueryUI's built-in behaviors often aren't sufficient for your needs, so you wind up writing a bunch of custom code to handle additional bindings and behaviors. That code winds up being full of widely-scoped methods and tight coupling between objects, and this quickly becomes unmaintainable. The Eos front-end certainly wound up like this.

I don't want to develop web applications that way any more. I'd like to try something which looks more like this:

  • The purpose of a web page is to be a container which provides context (URL, session, DOM, pub/sub broker, etc) for the items within it. It has little or no built-in functionality as initially sent down to the browser, and only enough markup to establish a basic layout according to the design.
  • Based on the context provided by the page, the browser's javascript parser instantiates one or more objects. These objects are responsible for generating their own markup and inserting it into the DOM, retrieving whatever data they require from whatever service they need (page context is important here), and establishing their own event bindings. They communicate with each other (and even other resources not on the page) using a pub/sub/multicast RPC-like asynchronous messaging API (JSON-RPC is a good choice of protocol here).

When viewed like this, it seems that the development process is very different than the one I've shown above. In particular, it forces front-end devs to think in terms of components and interfaces from the beginning. I think the process would look more like:

  1. Start with the design of an entire site and figure out what components you're going to have to build. Many of those components will exist in multiple places and will appear and/or behave differently depending on the page context (a simple example would be a navigation tool which needs to know the URL or session information so it can determine how to render itself).
  2. Define each component's interface: what messages it will send in response to what events, how it will respond to any messages it has subscribed to, and the exact format of each of those message.
  3. Code the pages for site, including their basic markup, look/feel (CSS), etc. At this point the pages will be mostly empty, with no functional components.
  4. Code the javascript objects and markup representing those components, with their event bindings, methods, properties, and insert them into a library through which they can be loaded individually into a page and instantiated via a Constructor or Prototype pattern.

(Steps 3 and 4 could be done at the same time if you have some programmers who are better at doing layout/look/feel and others who are better at javascript)

It seems to me this architecture of loosely-coupled components interacting through a common message-based framework has some advantages:

  • By focusing on architecture first, we're likely to spot problems with the design earlier than later. In this model a front-end architect's job will largely be to define those interfaces.
  • A message-based front-end architecture which uses well-defined interfaces and pub/sub means unit testing is not only possible, but easy. Front-end test-driven development, yo!
  • Components which are reusable are going to encourage the use of more semantic markup, which *should* lead to clearer CSS and improved maintainability.
  • This message-based model currently exists as a way to handle inter-frame browser programming. Take a look at http://code.google.com/p/pmrpc, it's basically a set of wrappers around HTML5's postMessage API which implement an RPC model using JSON-RPC. It provides a framework for callbacks, timeouts, retries, synchronous and asynchronous procedures, etc. But the thing I'm most interested in is the multicast/publish-subscribe stuff, along with service discovery. The project is designed for inter-window communication, but the ideas there make sense even within a single window context.
  • A service-oriented application architecture should be more flexible, allowing right out of the gate the development of apps on different platforms (mobile, etc). It also divides the tasks sensibly between front-end developers and application developers.


My guess is that at this point, you're either going "welcome to 2007" or "Pete just discovered IOC." OK, but while it's true that there are some shops doing development using models very similar to this, and everyone is practicing at least some level of componentization and code reuse, the vast majority of web applications I've seen still use some version of what I described at the beginning of this note, and I can't think of any applications that use intra-window pub/sub messaging interfaces between UI components.

Why is that? The tools for doing this have been around for a while--you don't need HTML5's publishMessage API to do pub/sub (although it certainly makes it easier). I've been dynamically inserting markup into the DOM since at least 1999, and jQuery has made event-based programming almost trivial (take a look at http://api.jquery.com/category/deferred-object/ and consider what you can do with it). Something like Google Web Toolkit allows us to take code generation to the extreme.

I think the real reason has nothing to do with programming. If you look at how designers work, they've traditionally thought mainly in terms of pages and less in terms of holistic application behaviors. If you think about it, the fact that a designer thinks it's acceptable to deliver a single screen at a time is astounding--they're assuming that what they've designed on that screen will be appropriate for the rest of the application, a huge assumption indeed. But it's understandable given that 15 years ago designers and design firms got most of their work from print and ad design, which focus on creating standalone images. That design culture has persisted in front-end software development, where most front-end software developers (myself included) have little or no formal training in object-oriented software design, and many of them are in fact visual designers first. So they start with the visuals.

And building things this way is tremendously appealing, because--at least initially--it seems like you're making a *ton* of progress. With a few hours work a competent front-end coder can have something on the screen which looks exactly like the most complex mock. But it's an illusion--it doesn't *do* anything yet. The real work comes when you start coding behaviors and at that point two things always happen: first, you try and reuse some component from elsewhere in the app, but find it too tightly-coupled to that other context to be useful, so you build another version just for this page; and second, as the designers and product managers create more pages they uncover invalid assumptions and fix them by changing the design. You find yourself chasing a moving target.

So how to fix this? I honestly don't think we ever will--I've never worked on anything that wasn't a moving target, and it's probably a good thing that we have designers who want to code user interfaces. But what I've proposed here at least offers some hope of dealing with those changes in a way that makes sense. If an architecture forces us to think first about discreet, reusable browser components, we're more likely to confine our changes to those components--and perhaps stimulate some thinking about consistency of design in the product itself. By relying on well-defined interfaces and a message-based architecture we ensure that our components stay testable as we make changes.

Are there problems? Of course. Componentization and code reuse have been fundamental tenets of software development forever and we still screw those up as often as we can. There's nothing to stop a proliferation of similar components and interfaces--and for that matter, web services--when refactoring a single component to handle several different contexts would be a better choice. But *starting* with the components and their interfaces rather than whole pages at least encourages reuse more than the way we're doing it now.

Also, there are a number of other obvious issues:

  1. HTML5 is widely but not universally supported. I don't think this is a big deal--the postMessage API is supported in IE8+, FF3+, Opera 9 and all webkit browsers since at least 2008.
  2. Dynamically generated markup isn't currently parsed by the googlebot. I honestly don't know what to do about this one, but in an increasingly services-based-architecture world they're going to have to do something about this eventually.
  3. Figuring out how to dynamically generate and cache javascript and load it with performance in mind is non-trivial, especially when you consider that an attractive way to organize this kind of application is by separating components in the filesystem, each with its own markup and CSS. How about a javascript buffer into which objects are concatenated at runtime, and which is then linked as a single file in the page markup? Ditto with CSS.

I'm sure there's more, but this seems like the right place to stop and start coding the "hello world" for this. If you've got this far, thanks for reading, and I'd love to get any feedback you have, pointers to projects doing this kind of thing, etc.