December 14, 2011

Displaying Service-Based Data, Part 1

Developers are frequently tasked with creating systems for the visual display of quantitative information (http://www.edwardtufte.com/tufte/books_vdqi). Over the years I've done a lot of these, and I've developed some ideas about what works and what doesn't. This is the first of several posts on the subject.

To start, I think it's important to recognize that there are three fundamentally different steps in the process of getting information displayed in a web browser:
  1.  Data retrieval
  2.  Data formatting
  3.  Data display
Historically, most information which eventually made it to a browser was either:
  •  Sent down to the browser as as part of a web page
  •  Assembled (or proxied) by the same server which served the page itself and exposed as a web service
  •  Retrieved and displayed in a separate frame
There are still good reasons to use the first two options, including bootstrapping, caching, removal of sensitive data, and complex business logic which may be unavailable to the browser. There is never a good reason to use the third option.

Prior to the widespread adoption of server-side support for JSONP the same origin policy meant that most web applications relied on their own servers for data services. This is no longer the case; the web developer is increasingly free to retrieve data from web services directly. From my point of view this is a fantastic development for two reasons:

  1. It frees companies to focus on their strengths. While Facebook, the USGS and Google all have browser interfaces, the most interesting work is being done by others who use the data made available through those organization's APIs. 
  2. It much more closely aligns the skillsets of the various software developers who work on a product with the parts of the stack where they work. Java devs are free to concentrate on ORMs without worrying about display logic. Front-end devs have access to those data models and they can use them to quickly build dynamic user interfaces.

Having spent 7 years working in a Java/Velocity framework (and JSP before that--ouch!), I'm particularly keen on the second point. I've always felt that the arguments about how much rope a templating language should give you reflect the fact that server-side templating languages are a hack in the first place. They're an unfortunate byproduct of early browsers and ECMAScript1 forcing us to move a big chunk of display logic onto the server.

In an earlier post I talked about how the big problem in front-end development is getting developers build their applications in the right order. In an environment where all the display logic is handled in the browser, creating a framework and a workflow which supports this is critical.

An Example

Say you were building an application in which you needed to display the current temperature in San Francisco. You'd have the option of using any of a number of publicly available APIs.

Here's the one from the Weather Underground's Weather API:

http://api.wunderground.com/api/d73bc72d0f231c10/conditions/q/CA/KSFO.json

Here's the same request from the Yahoo Weather API:

http://query.yahooapis.com/v1/public/yql?q=select%20item%20from%20weather.forecast%20where%20location%3D%2294107%22&format=json

Note how differently those requests are formatted.

Now take a look at the responses. The one from the Weather Underground looks like:

{
    "response": { ... some stuff ... },
    "location": { ... a whole bunch of stuff },
    "current_observation": {
        ... a bunch of stuff ...

        "temp_f": 55.8,

        ... a bunch more stuff ...
    }
}

... while the one from Yahoo! looks like:

{
   "query":{
      "count":1,
      "created":"2011-12-14T21:57:51Z",
      "lang":"en-US",
      "results":{
         "channel":{
            "item":{
               ... a bunch of stuff ...

               "condition":{
                  "code":"30",
                  "date":"Wed, 14 Dec 2011 12:55 pm PST",
                  "temp":"56",
                  "text":"Partly Cloudy"
               },

               ... a bunch more stuff ...
            }
         }
      }
   }
}

Note how differently these responses are formatted. If you were absolutely sure about which one you were going to use, you could just slam this data into a javascript object and start working on displaying the data, but that's almost never the case. Your company may decide to switch providers, or you may need to support multiple providers or combine data sources.

A much better solution is to create a plugin with two parts: a request URL formatter and a data normalization function which maps the response to common format--one that makes sense in the context of your application. Here's a simple example:

myApp.plugins.wunderground = {

    url: function(station_code){
      var base_url = "http://api.wunderground.com/api/d73bc72d0f231c10/conditions/q/";
      return base_url + station_code + ".json?" + "callback=myCallback";
    },

    formatData: function(response){
      var formattedData = {};
      formattedData["id"] = response.current_observation.station_id;
      formattedData["temp"] = response.current_observation.temp_f;
      return formattedData;
    }
};

Once you have that in place, the process of displaying data becomes independent of the data source. I'll talk about why that's important in the second part of this series.

No comments:

Post a Comment