WhitherApps

BBC News, pt IV

James — Tue, 05 Oct 2010 01:06:15 +0000

OK, after yet another long gap, I’m back. But the hiatus was worth it. I’m excited to announce that I’ve a new job & will soon be working with those smart chaps at Sencha… so hopefully there will be lots more of this sort of thing to come.

We promised ourselves that we would have fun with some CSS and get this application looking a little smarter. It certainly isn’t as glossy as the real thing yet. So here’s our mission statement for this installment: “indistinguishable or your money back”.

We have all the structural areas of the page in place, and the basic user experience – click a thumbnail, get an article. So there should be enough there to be able to style up.

Before we start: one caveat. I’d quite like to see if we can avoid having to use any external resources for this app – apart from the BBC material itself of course! In other words, every button, gradient and shadow you see has to be articulated within our single HTML page. Fortunately for us, the WebKit support for CSS3 styling (or at least, its -webkit- variants) gives us a reliable way to do most, if not all, of this. Let’s see.

Let’s start at the top of the page, where we have a dark red bar across the top of the app. Until now, it’s been styled with:

background:#ff0000;

…which is not quite the craftsmanship I have in mind.

I’ve taken a crop of the top of the application so we can do a proper analysis of it. (Is it me or is the BBC logo not quite centered with the clock?)

The #bbc_controls box is 44 pixels high, which is the upper pink border all the way down to the dark burgundy shadow. The main red part of it has a gentle gradient. It would be very easy to crop this image and use it as a background, but let’s see if we can do it with some CSS. First we need to sample the colors used. To start with, I’ll just use the Apple Digital Color Meter tool to sample some key colors.

The RGB of the pink border at the top is #e36d51. The two lines of shadow at the bottom are #7c110f and #37080a. The red in between goes from #bd1c08 at the top down to #990700 at the bottom.

As an aside… is the gradient linear across the middle 41 pixels of the bar? The histogram for the gradient looks like this:

Apparently not. There are 10 pixels at the bottom of the bar that are the same color (#990700) and the gradient is only at the top of the bar. Even that is not quite linear, but let’s pretend it is for now, yes?

So anyway, let’s do the CSS for this bar. I’m going to use WebKit’s -webkit-gradient CSS property, which is an alternative to having an image URL for the background. I’ve left the largest slab of color (#990700) as the explicit background-color. At least non-WebKit browsers will have something nice to show.

So, as a first stab:

background-color:#990700;
background-image:-webkit-gradient(linear, left top, left bottom,
  from(#bd1c08), to(#990700)
);

This provides a smooth gradient from the top of the bar to the bottom:

But we need to add some color stops that tell the property that the gradient only goes about two thirds of the way down. Color-stops, unfortunately, are seemingly only specified in percentage, rather than pixel distances. We want the gradient to stop 11 pixels from the bottom of the bar: handily this exactly 75% of the way down.

background-image:-webkit-gradient(linear, left top, left bottom,
  from(#bd1c08), color-stop(0.75, #990700), to(#990700)
);

While we are at it, we might even be able to use the gradient stops to do the highlights and shadows at the top and bottom of the bar. Each pixel is 1/44 (2.27%) of the bar. So…

background-image:-webkit-gradient(linear, left top, left bottom,
  from(#e36d51), color-stop(0.0227, #e36d51),
  color-stop(0.0227, #bd1c08), color-stop(0.7500, #990700),
  color-stop(0.9545, #990700), color-stop(0.9545, #7c110f),
  color-stop(0.9772, #7c110f), color-stop(0.9772, #37080a),
  to(#37080a)
);

Doubled-up color-stops allow you to disjoin the gradient. So we have a one pixel line at the top, then the gradient, then the solid slab, and then two distinctly-colored lines at the bottom. Obviously these percentages only work when the height of the bar is precisely 44 pixels, but we’ve ensured that elsewhere in the CSS.

How does it look?

As they say in New Zealand: sweet as.

The gray ticker bar beneath the top bar doesn’t have a gradient, but it does have a white highlight at the top. We can just do that with a border (and shrink the box’s height by one pixel to account for it:

#bbc_ticker {
  ...
  height:30px;
  border-top:1px solid #ffffff;
}

Before we move on to more gradients and coloring in the selector panels, let’s take a look at some of the typography. We haven’t specified any fonts yet, and it certainly shows. Now, I’m not the world’s greatest font spotter, but there aren’t that many native on the iPad, so figuring out which to use shouldn’t be that hard. We’re aiming for:

The caps are just about 9 pixels high. (It makes it easier to check you’ve got the right font when you’re comparing like sizes). I thought it was Arial at first, but Helvetica seems to be more or less the best match. The fonts seem to be the same on both the section headings (h2) and article headings (h3), except that the former is capitalized. Of course we can do that in CSS too – and we also tweak the kerning and spacing slightly to make the fonts line up as closely as possible with the original.

#bbc_article_selectors * {
  font-family:Helvetica, Arial;
  font-weight:normal!important;
  font-size:12px!important;
  letter-spacing:0.02em;
  line-height:16px;
}
#bbc_article_selectors h2 {
  display:none;
  text-transform: uppercase;
  line-height:8px;
  padding-top:7px;
}

Now is a good time to style up those thumbnail boxes too. They have a fine gray line around them (#464646 on 3 sides, #595959 on the bottom) with a very slight radius, and a background of #222222. There’s a small inset drop shadow. The images themselves have a faint #3A3A3A border around them.

Here’s something very annoying: the images that come from the BBC are 105px by 79px: which is exactly the size that they appear on the app – but with the border included. Should I shrink the image down 2 pixels in each direction so that the border makes up the extra? Or is there a CSS3 way of placing an image border *over* the edge of an image? I don’t know one, so I’ve had to slightly shrink the image to fit the box. The CSS now looks something like:

#bbc_article_selectors nav a {
  display:inline-block;
  width:105px; height:127px;
  margin:4px 5px 6px 5px;
  padding:5px;
  overflow:hidden;
  background:#222222;
  border:1px solid #464646;
  border-bottom-color:#595959;
  -webkit-border-radius:2px;
  -webkit-box-shadow: inset 0px 0px 2px #000;
}
#bbc_article_selectors img {
  border:1px solid #3A3A3A;
  width:103px;
  height:77px;
}

And the result is below. One is the native app, and one is the web app. Can you tell which is which?

Oh, and hey – I feel like doing the logo:

BBC News

#bbc_logo {
  position:absolute;
  left:446px;
  top:14px;
  line-height:18px;
  font-family:"Gill Sans";
  font-size:19px;
  text-transform:uppercase;
}
#bbc_logo .bbc_bbc {
  display:inline-block;
  width:18px;
  height:18px;
  text-align:center;
  margin-right:3px;
  background:#ffffff;
  color: #990700;
}

Might not get past the strictest of brand police, but pretty close, no?

And we also need to do the refresh button. I was nearly stumped here, until I discovered the Unicode character ‘CLOCKWISE OPEN CIRCLE ARROW’ (x21BB since you ask). This looks nice in the Arial font. Throw in a few shadows and gradients:



#bbc_refresh {
  position:absolute;
  display:block; width:38px; height:24px;
  left:142px; top:10px;
  line-height:18px;
  text-align:center;
  font-family:"Arial";
  font-size:30px;
  text-decoration:none;
  color:#ffffff;
  border:solid 1px #880600;
  background-image:-webkit-gradient(linear, left top, left bottom,
    from(#c20c00), to(#a10800)
  );
  -webkit-border-radius:2px;
  -webkit-box-shadow: inset 0px 1px 1px #ff6666, 0px 1px 1px #bc0e05;
  text-shadow:0px -2px 1px #880600;
}

And we have:

Nice. Not too bad for an imageless element.

OK, I think I’ll leave it there for today – might just be in time for Mobile Monday Silicon Valley, who are debating this very topic tonight. Progress so far:

Getting closer...

Next, we need to style up the articles themselves, and also sort out the portrait rotation. Back soon!

BBC News, pt III

James — Sat, 07 Aug 2010 23:51:13 +0000

There’s been a hiatus in our efforts to rewrite the BBC News app in HTML5 – no thanks to the odd tropical storm, a consuming day-job project, and kids who’ve decided that boomerang toucans take priority over their father’s Javascript hacking. So let’s get back to it.

We left things at the point where we had a static HTML skeleton that sort of looked like it could be the BBC News app if you squinted at it. Now it’s time to make it work.

To start with, we need to get our HTML file to be able to make calls to the BBC servers and retrieve JSON- and XML-based payloads. Immediately we run into a problem here, because browsers won’t make requests to different domains (or will, but probably won’t receive a successful response). I can’t upload my humble little HTML file onto the BBC servers, and of course I can’t get them to respond with JSONP or play the cross-origin preflight request game.

So I need to do a little server-side proxying of my own. It seems that Safari refuses to set the user-agent header in AJAX calls anyway – and if you recall from our earlier discovery, that’s important for this app. The proxy will be able to do this user-agent rewriting for us too.

Since I’m running whitherapps.com on an nginx server, running such a proxy is incredibly easy. I’ve simply added this to the server’s config file:

location /apps/bbc-news/proxy {
  resolver 8.8.8.8;
  proxy_pass $http_x_bbc_url;
  proxy_set_header User-Agent $http_x_bbc_ua;
}

…which basically means that any request to http://whitherapps.com/apps/bbc-news/proxy will be proxied on to the URL present in the X-BBC-URL header (as resolved by Google’s DNS server), and the user-agent will be rewritten to whatever is in the X-BBC-UA header. (Not using nginx for your own web server? Why not?)

In the Javascript for the app, I’d better provide a small AJAX handler that sets these headers appropriately and calls back to a provided success function (with the response text) or a failure function (with the XHR instance) as required. I also append a timestamp to the end of the URL, since I want to be sure that the browser doesn’t cache it, confused by my burying of the changing URL inside the headers.

var BBC_PROXY = "/apps/bbc-news/proxy";
var BBC_USER_AGENT = "BBC News 1.2.1 (iPad; iPhone OS 3.2; en_US)";
function bbcXhr(url, success, failure) {
  var xhr = new XMLHttpRequest();
  xhr.open("GET", BBC_PROXY + '?' + new Date().getTime(), true);
  xhr.onreadystatechange = function() {
    if (xhr.readyState == 4) {
      if (xhr.status == 200) {
        success(xhr.responseText);
      } else {
        failure(xhr);
      }
    }
  }
  xhr.setRequestHeader("X-BBC-URL", url);
  xhr.setRequestHeader("X-BBC-UA", BBC_USER_AGENT);
  xhr.send("");
}

(If I was the BBC and wanted to do this properly, I would simply need to alter this function and/or accept requests from other user-agents, and/or respond to cross-origin requests on these URLs… confusingly the app needs to access a range of different domains.)

So, with the app-to-server communication more or less in place, let’s start using it.

If you recall from our original investigation into the app traffic, the first request is to a bootstrap URL that returns some helpful pointers in a JSON file. We need to call that when the app is refreshed (or indeed when it first loads up):

var BBC_BOOTSTRAP = "http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1";
function bbcRefresh() {
  bbcXhr(BBC_BOOTSTRAP, function(responseText) {
    var bootstrap = JSON.parse(responseText);
  }, function(xhr) {});
}

addEventListener("load", function() {
  bbcRefresh();
}, false);

While we are at it, let’s create a crude refresh button in the control bar at the top of the app. That should help with testing things:

[Refresh]

And in the page load event, add a handler for it:

document.getElementById('bbc_refresh').onclick = function() {
  bbcRefresh();
  return false;
};

Put those in our app file, and… yes, it works: I get the slab of JSON back, and it’s parsed into the bootstrap object by the JSON parser available in Safari – which, although I suppose I do trust the BBC, feels far safer than eval().

Now, there’s a bunch of interesting things in that structure, such as links to privacy URLs and the like, but for now, we’ll just iterate through the list of feeds. These will populate the sections of the UI.

I’ve noticed that the real BBC News app loads the news from the top six or so categories preemptively, but when you click on the other, slightly more obscure categories (like the one for that little-known place, “Europe”), it goes and fetches them on-demand. I presume this is the default=true property set on the major feeds and that the other feeds just don’t work if you’ve never opened them and then go offline.

Rather than taking the feed information and creating the UI elements directly, I’ve decided I’ll do it in two logical steps. First get the feeds and stash the data about them in a local data store, and then secondly construct the UI from the contents of the data store. This is primarily to cater for users who go offline during the life of the app, or who start it up in offline mode and want to see what was there last time.

In fact, now I think about it, the easiest way to do this might be to wrap the local storage functionality in and around the AJAX function. We’ll key data in the browser’s local datastore off the URL. If the device is online, the browser gets fresh data, otherwise it gets it from the local storage.

So here’s a slightly beefier AJAX function that makes this happen behind the scenes:

function bbcCachedXhr(url, success, failure) {
  var xhr = new XMLHttpRequest();
  xhr.open("GET", BBC_PROXY + '?' + new Date().getTime(), true);
  xhr.onreadystatechange = function() {
    if (xhr.readyState == 4) {
      if (xhr.status == 200) {
        localStorage.removeItem(url);
        localStorage.setItem(url, xhr.responseText)
        success(xhr.responseText);
      } else {
        responseText = localStorage.getItem(url);
        if (responseText != null) {
          success(responseText);
        } else {
          failure(xhr);
        }
      }
    }
  }
  xhr.setRequestHeader("X-BBC-URL", url);
  xhr.setRequestHeader("X-BBC-UA", BBC_USER_AGENT);
  xhr.send("");
}

It still calls either a success or failure function, but the former will be called even if the AJAX request is unsuccessful (for whatever reason) but the data was still in the local storage.

This approach works for the BBC app because there are a relatively small number of URLs in play: all the articles for a given section are in one single structure at a stable URL. If there had been a URL for each article, I might have been worried about the cache filling up after a few weeks… but this seems OK for now. This week’s 10 articles will simply overwrite all of last week’s, since they are keyed off the same URL.

I have an idea that this function will be very useful both here and in future WhitherApps project.

Right then. Let’s take the feeds (either just fetched, or rescued from local storage), fetch the articles and create the UI. In part 2, we set up some dummy category panels. Let’s remove them from the static HTML and simple leave their containers:

So we need a function that creates all the pieces needed for a given feed category selector. There are two titles (one which appears in the portrait mode, and one in landscape), and the selector panel itself. This function creates all three – if they are not present – and returns the main selector panel.

function bbcEnsureSelector(title) {
  var id = title.replace(/[^a-zA-Z0-9_]/g, '_')
  if (document.getElementById(id+'-t1') == null) {
    var selectorT1 = bbcCreateNode('a', id+'-t1', title);
    document.getElementById('bbc_category_selector').appendChild(selectorT1);
  }
  if (document.getElementById(id+'-t2') == null) {
    var selectorT2 = bbcCreateNode('h2', id+'-t2', title);
    document.getElementById('bbc_article_selectors').appendChild(selectorT2);
  }
  var selector = document.getElementById(id)
  if (selector == null) {
    selector = bbcCreateNode('nav', id);
    document.getElementById('bbc_article_selectors').appendChild(selector);
  }
  return selector;
}

function bbcCreateNode(tag, id, innerText) {
  var node = document.createElement(tag);
  if (id!=null) {
    node.id = id;
  }
  if (innerText!=null) {
    node.appendChild(document.createTextNode(innerText));
  }
  return node;
}

This may look a little clumsy, but it does include making a safe unique ID (from the title) and using DOM methods to ensure the resulting markup structure is well formed. (I’m thinking that a few of these functions might get moved into a separate library at some point.)

Then, as just after we parse the bootstrap in the refresh function, let’s call this for each feed:

var feeds = bootstrap.feeds[0].feeds;
for (var f in feeds) {
  var feed = feeds[f];
  var selector = bbcEnsureSelector(feed.title);
}

Seems to work (“At last! A screenshot!”):

Category titles appearing in the left sidebar

Now, each of these category selector panels need to have the article thumbnails placed in them. This should happen when the bootstrap is loaded (for those that have default=true in the bootstrap, it seems), and also when a selector is shown that hasn’t previously been populated. So we’ll do it in a separate function that can be called in either case.

function bbcPopulateSelector(selector, url) {
  bbcCachedXhr(url, function(responseText){
    var feed = new DOMParser().parseFromString(responseText, 'text/xml').documentElement;
    if (feed.nodeName=='feed') {
      var feedId = feed.getElementsByTagName('id')[0].textContent;
      var entries = feed.getElementsByTagName('entry');
      for (e=0; e < entries.length; e++) {
        entry = entries[e];
        var entryId = entry.getElementsByTagName('id')[0].textContent;

        var id = feedId + '-' + entryId;
        var thumbnail = bbcCreateNode('a', id+'-thumbnail');

        var thumbnailImage = bbcCreateNode('img', '');
        thumbnailImage.src = bbcRewriteMediaUrl(
          entry.getElementsByTagNameNS('*', 'thumbnail')[0].getAttribute('url'),
          entryId
        );
        thumbnail.appendChild(thumbnailImage);

        var thumbnailTitle = bbcCreateNode('h3', null,
          entry.getElementsByTagName('title')[0].textContent
        );
        thumbnail.appendChild(thumbnailTitle);

        var content = entry.getElementsByTagName('content')[0];
        var contentNodes = content.childNodes;
        var article = bbcCreateNode('div', id+'-article');
        for (var i=0; i < contentNodes.length; i++) {
          article.appendChild(document.importNode(contentNodes[i], true));
        }

        thumbnail.article = article;

        selector.appendChild(thumbnail);
      }
    }
  }, function(xhr){});
}

function bbcRewriteMediaUrl(url, base) {
  if (url.substr(0, 11)!='bbcimage://') {
    return url;
  }
  url = url.substr(11);
  url = decodeURIComponent(url);
  if (url.substr(0, base.length)!=base) {
    return url;
  }
  url = url.substr(base.length);
  url = url.replace('{device}', 'ipad');
  return 'http://' + url;
}

So what’s going on here? Firstly we call the feed URL (again, cached via localStorage if offline), and then parse the results. For each article entry in the feed, we get its ID, and create a thumbnail link that will sit in the article-selecting category tray. We use the thumbnail image to actually get a picture for it (using the bbcRewriteMediaUrl function to transform its rather strange syntax, as observed in part I), and add the title of the article beneath it. We have to use getElementsByTagNameNS to get the thumbnail image URL, due to the media RSS namespace on that element.

Then, embedded in the Atom is a slab of HTML under the element for the article itself. We get its children out and lash them into a container held under the thumbnail’s DOM using the ‘article’ attribute. I’m not entirely sure if this is the most kosher way to do things, but it will do for now. Basically we want it in a place where we can switch it into the main reading panel whenever someone clicks on – or rather, touches – the thumbnail.

After the selector panel is known to be in place in the refresh sequence, let’s call this with URL of every default feed. We have to use ‘default’ as an array index when examining the feed object, rather than an object property, because it’s a reserved word in Javascript and Safari complains otherwise:

if (feed['default']) {
  bbcPopulateSelector(selector, feed.feed_url);
}

OK, so let’s run that up and see what happens. Hopefully, the bootstrap and then the default feeds will be fetched. We should then see a panel appear for each of the categories, and the images and titles of the articles within them. And…

The thumbnails ...not looking quite right

Well, OK. It did what I claimed, more or less, but we obviously have a few cosmetic problems here. Our category trays are of a constrained height, and the thumbnails are not at all styled to fit them. But it shouldn’t take a lot of CSS to sort this out. Firstly, let’s sort out the tray overflow, and then the display layout of the thumbnails themselves. (We’ll deal with making scrolling look nice in due course – the iPad doesn’t show scroll bars for overflow:scroll, although you can use two fingers to move the trays left and right).

So let’s make the following small additions to the CSS:

#bbc_article_selectors nav {
  ...
   overflow-x:scroll; overflow-y:hidden;
   white-space:nowrap;
}
#bbc_article_selectors nav a {
  display:inline-block;
  width:115px; height:137px;
  margin:5px;
  white-space:normal;
  overflow:hidden;
  background:#000000;
  border:1px solid #ffffff;
}

What a difference that makes. Arnie looks a bit odd (which is surely setting somebody up for a punchline) for an as-yet-unknown reason… but most of the thumbnails now look respectable. We’ll sort out the typography soon enough, but even with the default

styling we can still read most of the headlines.

Thumbnails looking a little better

Of course we should make these thumbnails actually do something. After all, we have the text of the article hanging off an expando DOM property, so let’s put it to good use. We can bind an article display function onto its click event at the end of the bbcPopulateSelector function. It’s tempting to use an anonymous function here, but I think I’ll need this in other places eventually (since in the app you can swipe articles to change them too).

...
  thumbnail.onclick = bbcDisplayArticle;
...

function bbcDisplayArticle(event, thumbnail) {
  if (thumbnail==null) {
    thumbnail = this;
  }
  var articlePanel = document.getElementById('bbc_article');
  articlePanel.innerHTML = '';
  articlePanel.appendChild(thumbnail.article);
  return false;
}

Crude for now, but this latter function will be a good place to put in some swanky page sliding animation at some point.

So at least we can now pull up articles:

Houston, we have an article

Lovely. Seems to be one final thing to do though: rewrite the images that appear in articles. It makes sense to do this once – probably at the time we are stealing the article out of the Atom DOM and putting it into the HTML one. Let’s just do a quick scan for tags and rewrite their src attributes, just before the importNode function puts them into the DOM (this means that Safari won’t try to go and get them before we can get them into the right syntax).

var contentImages = content.getElementsByTagName('img');
for (var i=0; i < contentImages.length; i++) {
  contentImages[i].src = bbcRewriteMediaUrl(contentImages[i].src, entryId);
}

It turns out that some of the images are placeholders for BBC video files too: these are tags with bbcvideo:// schemes and bandwidth metadata. Let’s look for those too, and enhance our media URL rewriter:

var contentVideos = content.getElementsByTagName('a');
for (var i=0; i < contentVideos.length; i++) {
  contentVideos[i].href = bbcRewriteMediaUrl(contentVideos[i].href, entryId);
}

if (url.substr(0, 11)!='bbcimage://' &&
    url.substr(0, 11)!='bbcvideo://') {
...
   url = url.replace('{bandwidth}', 'wifi');
...

There’s more to the video thing than meets the eye. If you try the app you’ll be confronted with a cryptic MP4 URL in a manifest of some sort, and so I’ll need to do a bit more hacking to get the videos showing. Something to do with British TV license payers or some such nonsense. But at least we now have images within the articles:

Articles with pictures (just don't click on the video links!)

And, if you compare it with a screen shot of the real thing, you’ll see we’re actually not doing too badly:

The real thing, earlier today

That’s it for this time. But we have an app you can just about use. Please check it out on your iPads here (but don’t all go too crazy at the same time, since it’s using my server as the proxy!) and see what you think. Yes, I know the portrait version doesn’t work yet, and the scrolling is pretty poor… but hey, come on.

In the next episode, after all this heavy lifting, I think I deserve to play around with some CSS. Lots of lovely -webkit-gradient coming up (I think that’s what you were all waiting for, anyway).

Back soon.

BBC News, pt II

James — Tue, 22 Jun 2010 05:21:47 +0000

In part two of our attempt to rewrite the BBC News app with web technology, we’re going to move onto the page structure itself. Some of you may have been left cold by last week’s trawl through what was happening behind the scenes, so let’s get up front again, and try and at least capture the main UI elements of the app in both portrait and landscape mode.

At this point, I am going to work with a single static HTML file. I’m not going to be pulling any dynamic content yet, and nor am I going to confuse things by trying to get too clever too soon. Hopefully a simple file will make it easier to understand my thought process.

The first thing to notice when looking at the app is that the landscape/portrait rotation changes the layout quite considerably. So most of what we’ll try to do first is understand how that can be done elegantly with a bit of CSS and maybe Javascript. Although the page as a whole looks quite different, the same panels are more or less present, so it’s more a matter of moving things around than changing functionality fundamentally.

Page topography

If you have the app open in front of you, take a look at how it’s laid out.

The BBC News iPad app in portrait mode

The portrait mode of the app is basically a series of horizontal panels, starting with the red control bar at the top. It has a 1px highlight at the top, 41px of slight gradient red, then 2px of dark shadow at the bottom. Beneath this is a grey bar continent the news ticker. It also has a 1px highlight at the top and then 30px of solid grey. Then there is the category selector (‘Top Stories’, ‘Americas’ and so on), which is 36px of solid black. After this is the news story selector: 152px of dark gray, with a 1px border at the bottom. Finally we’re into the page itself, 614px high, and scrollable. At the end of the page is a fixed 90px ad (although not always), and a 36px footer, containing a horizontal scroll indicator.

(Let’s just check all that adds up. 1,004px, which, yes, with the 20px iPad bar at the top of the screen gives us the device’s full 1024px).

The BBC News iPad app in landscape mode

In the landscape mode, we have a slightly more interesting layout. The control bar and news ticker are exactly the same as the portrait mode (although, of course, wider), followed by a 1px black rule. After that, on the left of the screen, 507px wide, we have the full height of the screen dedicated to combined category and news story selectors. These are 168px in height each, with a single 1px shadow underneath. Below these ‘open’ panels are the headings of the other news sections. These include red bars (18px plus 2px highlight above and 2px shadow below) and larger collapsed panels (40px plus 2px highlight above and 2px shadow below). The very bottom of the left hand pane is a hatched area of 106px containing copyright and help links. Between the left and right panes is a single pixel grey separator, and then we have the main news page, 516px wide by 636px high. The 36px footer rounds off the right hand side of the page.

It’s interesting to note that the designers have actually made the main reading pane narrower in the landscape mode than in portrait.

At first blush, neither page layout alone seems too hard to reproduce with HTML and CSS. The challenge will probably be moving and removing portions around when the device rotates between the two.

Some markup

Let’s start with the outer part of the page’s HTML. We want to make sure it’s nice and standard, and declares viewport settings correctly.



 
  BBC News
  
  
 
 
  ...

The apple-mobile-web-app-capable setting in the first tag removes the thick browser navigation bar from the page when the user has added it as a bookmark on their home screen. This will give us exactly the same real estate as the native client app (i.e. the whole page with the exception of the black 20px bar at the top). The second tag disables the pinch and zoom scaling on the page, which is not present on the app we are trying to emulate.

Taking the portrait view first, we’ll create some nice semantic panels for the structure of the page:

BBC
News ticker ...
Top Stories, Americas ...
War, Pestilence, Famine ...
Oil, Budgets, Taxes ...
Blah blah ...
Buy ...
o o o o

You’ll see I didn’t spend too much time on the copy. We just want to get the right panels in the right place. Also notice that I use HTML id attributes to strongly identify these panels as they are unique within the app. The one exception is the article selector, which I have indicated with a class attribute because there are going to be many of these selectors (one for each news category) – certainly in the landscape mode where they are stacked on the left hand side, but also in portrait mode, where I think I’ll have multiple panels too, although only one will show at a time.

I suppose I could have used class attributes for all of the panels, but this seems sensible enough until the semantic police haul me in. Also notice that I prefix every id or class with ‘bbc_’. This sort of thing prevents collisions with any other styling or behaviors when you’re mixing and matching CSS and Javascript libraries. (Not that we’re likely to do so here, but I guess it’s habit).

Wait! This is HTML5. Let’s do something more interesting than using

tags.

BBC
News ticker ...
Top Stories, Americas ...
War, Pestilence, Famine ...
Oil, Budgets, Taxes ...
Blah blah ...
Buy ...
o o o o

Many of our panels are predominantly there to provide navigation, so let’s promote them to be

tags. The main article can be an, er,

tag, and the ad (which is fairly auxiliary to the matter at hand) seems a good candidate for being an

. The footer is not a

BBC News, pt I

James — Sat, 12 Jun 2010 15:50:05 +0000

For the inaugural WhitherApps project, we’re going to look at the (excellent) BBC News iPhone/iPad app and see if it could have been built as a web app, rather than in native code.

To kick off with, we’re going to try and emulate the iPad version, in both orientation modes. What we’re basically aiming for is this:

The BBC News iPad app in landscape mode

And this:

The BBC News iPad app in portrait mode

Let’s see how we get on. I’ve decided to break the process down into 4 steps:

Figure out how the app pulls content from the BBC
Make a wireframe web page that behaves like the app
Stitch the content into the wireframe
Decorate the wireframe so it looks just like the real thing

This gives me a chance to assess the feasibility of the undertaking before I get too swept up with pushing pixels around.

Behind the scenes

Obviously the app does not ship with the news in it, and my assumption is that the app is little more than a client to render a feed of content and images located somewhere on the web. Architecturally, it’s nothing more than a customized browser-like client… but we do need to figure out where the data comes from and how we might be able to use it in a real browser-based app.

I’m using my iPad on a home WiFi network. I take a quick look at the network settings on the device and I see that I can set up a proxy for web access and so on. This seems like the easiest way to get in to see the traffic coming to and from the app: I run up a Squid proxy on my Mac (which is conveniently on the same local WiFi network) and set the iPad’s proxy to be my Mac’s local IP address.

As a proxy, Squid does lots of clever things. For our purposes though, we really just want to see the requests the device is making, so I tail the access.log file. This shows me the HTTP requests from the device as it makes them:

GET http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1 - application/json
GET http://www.bbc.co.uk/moira/feed/news_world/front_page - application/atom+xml
GET http://bbc.112.2o7.net/b/ss/bbcwnewsiphone/0/OIP-2.0/s82818894? - text/html
GET http://cdnedge.bbc.co.uk/nol/ifs_news/hi/front_page/ticker.json - text/javascript
GET http://www.bbc.co.uk/moira/feed/news_world/americas - application/atom+xml
GET http://static.bbc.co.uk/moira/img/ipad/thumbnail/48058000/jpg/_48058752_009512185-2.jpg - image/jpeg
...

This looks very promising. Firstly the app seems to be using HTTP – no proprietary protocols here – so this will work well if we come to use some sort of AJAX technique ourselves. Secondly, the URLs are self-explanatory, so it’s easy to see how things are working. There’s some sort of initialising JSON at the start, then an ATOM feed of the front page news (and shortly afterward the ‘Americas’ page), and then a whole bunch of thumbnails that are used for the navigation icons at the top (or side) of the app.

(Incidentally, if you do do this sort of thing, prepare to be intrigued by the amount of background HTTP that the iPad is sending to Apple!)

Playing it back

My first impulse of course is to fire up a web browser (or, in this case, wget on the command line) and see what the payload of some of these responses is. So I try the initial JSON file:

~ > wget http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1
--2010-06-13 13:13:50--  http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1
Resolving www.live.bbc.co.uk (www.live.bbc.co.uk)... 212.58.246.160
Connecting to www.live.bbc.co.uk (www.live.bbc.co.uk)|212.58.246.160|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2010-06-13 13:13:53 ERROR 403: Forbidden.

And this doesn’t seem so good. How does the iPad app get the content over HTTP (via my Mac) when my Mac itself can’t?

Well, like all self-respecting mobile technologists, I have something of a fetish for HTTP user-agents. I wonder how the iPad app identifies itself when it makes requests to the BBC server? I go back to Squid, alter the configuration slightly so that the HTTP headers are logged, and refresh the iPad app. A whole load of new HTTP goes past, but this time I can see that the app’s requests include:

User-Agent: BBC News 1.2.1 (iPad; iPhone OS 3.2; en_US)

Call it a hunch, but I wonder how wget on my Mac will get on if I spoof the user-agent header?

~ > wget --user-agent="BBC News 1.2.1 (iPad; iPhone OS 3.2; en_US)" http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1
--2010-06-13 13:18:20--  http://www.live.bbc.co.uk/moira/feeds/ipad/news/en/v1
Resolving www.live.bbc.co.uk (www.live.bbc.co.uk)... 212.58.246.160
Connecting to www.live.bbc.co.uk (www.live.bbc.co.uk)|212.58.246.160|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6199 (6.1K) [application/json]

Bingo! That’s all it takes to be not ‘forbidden’ and get our initial 6Kb’s worth of JSON. It looks like this:

{
 "name": "WWW iPad Application Bootstrap",
 "version": "1.0.4",
 "published": "2010-06-07 11:09:39 Etc/GMT",
 "ticker_url": "http://cdnedge.bbc.co.uk/nol/ifs_news/hi/front_page/ticker.json",
 "live_feed_uri_template": "http://www.bbc.co.uk/worldservice/meta/mobile/iphone/%7Bbandwidth%7D",
 "ugc_sms_number": "+447725100100",
 "ugc_email": "talkingpoint@bbc.co.uk",
 "feedback_email": "iphone-feedback@bbc.co.uk",
 "faq_url": "http://www.bbc.co.uk/moira/html/%7bdevice%7d/news/faq/en",
 "feedback_url": "http://www.bbc.co.uk/moira/html/%7bdevice%7d/news/feedback/en",
 "conditions_url": "http://www.bbc.co.uk/moira/html/%7bdevice%7d/news/tandc/en",
 "privacy_url": "http://www.bbc.co.uk/moira/html/%7bdevice%7d/news/privacy/en",
 "copyright": "BBC © 2010",
 "feeds": [
 {
  "type": "group",
  "title": "More",
  "feeds": [
  {
   "title": "Top Stories",
   "feed_url": "http://www.bbc.co.uk/moira/feed/news_world/front_page",
   "default": true,
   "movable": 0
  },
  {
   "title": "Americas",
   "feed_url": "http://www.bbc.co.uk/moira/feed/news_world/americas",
   "default": true
  },
...
  {
   "title": "Audio & Video",
   "feed_url": "http://www.bbc.co.uk/moira/feed/avod/iphone/news/en/v1"
  },
  {
   "type": "group",
   "title": "News in Other Languages",
   "feeds": [
   {
    "title": "Mundo",
    "feed_url": "http://www.bbc.co.uk/worldservice/syndication/mobileiq/iphone/mundo/homepage/full.xml",
    "logo_url": "bbcimage://logomundo/wsdownload.bbc.co.uk/worldservice/images/branding/languages/iphone/mundo_125x19.png"
   },
...
   {
    "title": "Urdu",
    "feed_url": "http://www.bbc.co.uk/worldservice/syndication/mobileiq/iphone/urdu/homepage/full.atom",
    "logo_url": "bbcimage://logourdu/wsdownload.bbc.co.uk/worldservice/images/branding/languages/iphone/urdu_117x28.png"
   }

   ]
  }
  ]
 }
 ]
}

This looks good. We can see that the first thing the app is doing is being told where all the critical feeds are stored, and how they should be structured in the navigational menu. We can see how to get the ticker data for the top of the page, and we can even get some ideas about how the ‘News in other languages’ will be fetched.

Let’s start by looking at the main feed for what must be the front page:

http://www.bbc.co.uk/moira/feed/news_world/front_page

Again, this requires the user-agent to be spoofed, and the response is as follows:



  BBC News | News Front Page | World Edition
  2010-06-13T20:29:08+00:00
  urn:news-bbc-co-uk:section:bbc_news:front_page:world_edition
  
    BBC
  
  
    Thousands flee Kyrgyzstan unrest
    Escalating ethnic violence in Kyrgyzstan that has killed nearly 100 people prompts tens of thousands to flee to Uzbekistan.
    
    2010-06-13T17:51:40+00:00
    urn:news-bbc-co-uk:story:8737578
    
    
    
      
        
          
            
          
        
        Escalating ethnic violence in Kyrgyzstan has prompted tens of thousands of ethnic Uzbeks to flee the country.
...
        
          
        
...
      
    
  
...

Excellent. This looks like a very straightforward ATOM feed, containing thumbnails for the navigation and fairly simple HTML formatting. The style of BBC articles is to have small, bite-sized paragraphs with small inline images and occasional videos. We’ll probably have to sort out all the styling ourselves, but the markup looks clean and workable.

The bbcimage://urn URLs for the images and thumnails look a bit strange, but we’ve already seen the HTTP traffic when they’re fetched, so we can figure out how they’ll need to be rewritten. At first glance it looks like:

bbcimage://urn%3Anews-bbc-co-uk%3Astory%3A8737578/static.bbc.co.uk/moira/img/%7bdevice%7d/thumbnail/48063000/jpg/_48063440_48063241.jpg

Will become:

http://static.bbc.co.uk/moira/img/ipad/thumbnail/48063000/jpg/_48063440_48063241.jpg

…which is a fairly simply transformation.

What next?

OK, so we’ve figured out how to bootstrap our app, get the structure of the navigation and news categories, and then receive some HTML-like content from the BBC feeds. So far so good: it looks quite simple, and certainly something that a self-respecting web app is going to be able to do.

The user-agent spoofing may not be possible from an AJAX-like context in a web browser, so that might require me to build a small proxy on a server somewhere, and means my web app will ultimately have to be more than a static HTML file. But then I suspected I would need to do that anyway, to avoid cross-site scripting issues, so that’s no big deal. (If the BBC were hosting the web app themselves, both reasons would be moot).

So, next installment, I’m going to be taking a look at the structure and user-interface of the app and see how easy it is to synthesize the same overall look and feel of it. Since I’ve seen the data in the background, I now suspect that much of the app uses Safari browser components anyway… so I’m quietly confident!

Stay tuned.

Introducing WhitherApps

James — Sat, 12 Jun 2010 14:25:54 +0000

The debate

It’s 2010 and debate rages in the mobile industry over the future of applications.

On one hand, people seem to love native client apps. A quick download process from an app store, and the shiny icon is sitting on the mobile device’s desktop, ready to go.

What’s more, as a developer, I might even get paid for it. But wait! Damn. If I want to reach lots of users, I have to write different code for every single device platform. iPhone, Android, Symbian. That’s a lot of code to keep in sync. And I need users to keep updating the applications every time I make a small change.

On the other hand, the mobile web. Build a site which looks and feels like a self-contained application but which can render nicely on a variety of platforms. (It helps that a lot of modern mobile operating systems have broadly similar browsers). I might not be able to make money by selling access to it, but I can update the site whenever I want, and users will see the latest and greatest version.

In reality, there’s an essential role to be played by both types of application – as there is for computing in general. But the debate still continues and it’s fun to be involved.

Enter WhitherApps

So what’s the point of this site, then?

WhitherApps is a bandwagon-busting experiment. I believe there are far too many native client apps which could have been far better written as mobile web apps. What we’re going to try and do is take a few examples, apply a little reverse-engineering, and rewrite them, warts and all, with web technologies.

We’ll focus on native client apps that are free to download. These wouldn’t have suffered commercially had they been written this way in the first place.

Obviously, we’ll also focus on feasible genres of apps. No 3D games or background apps, for example. To start with, informational media apps (such as newspapers and broadcasters) will be in the spotlight.

But otherwise, we may not need to pull too many punches. Offline access? Through the wonders of HTML5, that should be OK. Native device API access? Well, BONDI and proprietary libraries permitting, we’ll have a go – and there’s always PhoneGap too.

Each post on the site will be a walkthrough (or part of a walkhrough) of the process. As we get our hands dirty, from the protocol-sniffing to the pixel-jiggling, you can follow the steps we took.

Stay tuned and wish us luck!