The details of how fragment navigation (also hash navigation or hash fragment navigation) works have been covered extensively by others, and have been abstracted into various frameworks and toolkits. A number are available for jQuery, such as:

• History
• BBQ (tutorial)
• Address

However, fragment navigation comes with a new set of challenges that we found ourselves having to address, so these will be the focus of this post.

Full page alternatives

Rather than changing all links to the form <a href="#!fragment/path/for/ajax">Link</a> and then detecting and acting upon fragment changes in javascript, we wanted to ensure that our links worked when they were copied and pasted into a new browser window, shared by email, or used with javascript disabled. This meant making them into real URL links, and then progressively enhancing using a Javascript onclick handler to cancel the normal navigation action:

<a href="/path/works/as/fragment/or/normal/page" class="fragnav">Link</a>

And some jQuery to hook the onclick handler on these fragment-navigation-compatible links, and convert them so that rather than navigating to the specified href, the browser changes it’s hash instead:

$("a.fragnav").click(function() {
  var href = this.href.replace(/^https?:\/\/[a-z0-9\.]+\/(.*\#\!)?/, '');
  location.hash = this.href;
  return false;
});

Now you can either click the link with javascript enabled and get a dynamic AJAX behaviour, or right click then ‘Open in new window’ (or click with javascript disabled) to get the full URL of the link to load normally.

Caching

When your user presses ‘back’, you can detect the fragment change and reload appropriate content – great. But this is not as good as what you get from browsers’ normal back button behaviour. Normally, the previous page is cached, so it reappears almost instantly. In the AJAX version, unless your fragment is just switching between different versions of content that are all preloaded, you may fire an AJAX request to load the content again.

It shouldn’t be necessary to reimplement caching yourself, since AJAX requests are subject to the same browser caching as normal page loads, but it’s easy to forget that you should include appropriate cache headers with your AJAX responses. There are four HTTP headers that generally modify a browser’s caching behaviour:

• Cache-Control (Docs)
• Expires (Docs)
• ETag (Docs)
• Last-Modified (Docs)

Cache-Control and Expires are cache directive headers, telling the browser whether and for how long it may cache the resource. Typically we set only Cache-Control, as it is more powerful and flexible, and Expires is generally unnecessary.

ETag and Last Modified are validators. These allow the browser to make a conditional request to the server to find out if the resource has changed, and allow the server to respond with a simple 304 Not Modified if it hasn’t. It’s often the case that your web server will add these automatically, and we prefer to avoid them entirely, relying on a single Cache-Control header to determine browser behaviour.

Cache-Control: max-age:60, public

A nice trick for content that is personalised to an authenticated user is to include the private directive in your Cache-Control header, which allows the page to be cached, but only by the browser, not by any proxies along the way.

Scroll position

One of the things we found to be most difficult was managing scroll position. Consider the default behaviour of a browser when navigating normally. If you scroll down a page and click a link somewhere near the bottom, the browser loads the new page and displays it starting at the top. However, if you then press the back button, it redisplays the last page you viewed and restores your scroll position for that page. You may not have even noticed this, but if it stopped behaving this way you’ get pretty annoyed soon enough!

So, if a user clicks one of your fragment links low down on your page, and the change you make to the page as a result would appear to the user to constitute a new page (obviously the point at which perception of ‘a new page’ is triggered is subjective), consider scrolling the browser window back to the top.

But, you should not do this if the back button is used, because the browser will restore the user’s previous scroll position all by itself. However, this becomes more complex if navigating forwards has considerably shortened the page content, and restoring the scroll position on back would require you to restore the content first in order for the necessary scroll offset to actually exist. This is a bit confusing. Here’s an illustration:

So the solution we use is to remember the scroll position on every navigation action, and then restore it after repopulating the content if it looks like a backwards step. You need to ensure you have got your caching rules right to support this otherwise the delay in refetching the content will make the browser reposition the scroll position twice – once by itself immediately, and again triggered by your JavaScript after the content has loaded.

Loading pause

Normally, the process of clicking a link and navigating to a new page involves the current page remaining on screen while the new page is being requested. The browser only blanks it out when content starts to arrive for the new page. The browser does provide some progress feedback immediately though, in the form of a wait mouse cursor or spinner in the browser chrome.

We aimed to replicate this experience for our AJAX-loaded views. This means avoiding what seems like an obvious solution – empty the container when the link is clicked, and populate it when the AJAX response is received – because you’ll get a flash of blankness between the old content disappearing and new content replacing it. Instead, the old content should remain, and the new content should simply replace it when the AJAX completes.

But this doesn’t provide progress feedback. The risk is, the user will think nothing’s happened and will click the link again. This tends to happen after about 2-3 seconds for most users, but AJAX calls normally don’t take anywhere near that long. So we implemented a timeout that, after 250ms, would blank out the container, replacing the old content with a loading spinner. If the AJAX completes within that time, it cancels the timer.

So, in most cases the user clicks a link and sees an almost immediate cut from old content to new with no flash of blankness. In some cases they see the old content vanish and a loading spinner to reassure them that something is happening while we wait for the new content to come back.

If you regularly have edge cases where content takes more than a few seconds to load, you should consider moving on to a different kind of progress feedback, to avoid hitting that “it hasn’t worked” perception boundary. The aim is to keep pushing that boundary back, keeping the user’s expectations in line with the performance that they’re getting from the site. I like to think of this like a timeline:

By providing better progress feedback, you can expand the ‘Expectation’ phase and avoid the ‘Frustration’ phase. However, it’s also equally important that you don’t display a big piece of progress feedback for the entire operation to then complete in under 250ms – the feedback will be on screen for such a short period that the user will potentially be unsure about what happened and get anxious. So be aware of the ‘instant’ phase, when you should present no feedback at all.

Inline Javascript

Sometimes, it’s convenient to get both new HTML code and new Javascript when your user clicks a fragment link. In our case, we wanted to set some Javascript globals in the response to allow javascript code already loaded on the page to better understand the content that had just been loaded. Say the user has requested a page with a fragment that loads some search results. We also want to tell the browser the search query that produced the results so that it can cache them, or populate the query into a search field that’s outside the fragment container, or something.

Normally, if you load content from a server using AJAX and append it to the DOM using innerHTML, any <script> sections in the HTML are not parsed by the browser’s JavaScript engine. However, to make it do this is as simple as searching the returned source for <script> sections, and evaling them. Make sure you properly filter and escape end user input before allowing it to be evaled though, otherwise you’re opening up your site to XSS attacks.

$('#main script').each(function() { eval(this.text); });

Command, Control and Shift modifier keys

Power users like to open links in new tabs or windows. In fact when faced with a list of links I often hold down Control and hit each link to turn to start preloading all the results into tabs which I can then review in turn without having to wait for each one to load. To avoid annoying users, you need to ensure that whereever possible, control-click (or command-click) and shift-click (normally ‘new tab’ and ‘new window’ respectively) still work.

It’s easy to get lulled into a false sense of security by right clicking on your links and using the context menu to select ‘Open in new tab’. This won’t run your onclick handler, so most likely will work if your link has a non-JavaScript href. But using the keyboard CTRL key while clicking the link will fire the onclick event, and if you are cancelling the normal link navigation action by returning false, you’ll also be cancelling the new tab or new window request. Ensuring that you don’t is quite straightforward (remember to include e.metaKey to detect the Command key on Macs):

if (e.shiftKey || e.ctrlKey || e.metaKey) return true;

Focus rectangles

In most browsers, when you click a link, you get a small dotted rectangle around it. This focus rectangle also appears if you tab through the actionable items on a page, to indicate which one would be followed if you pressed enter. The problem is that if your link fires an AJAX action, and the link itself remains on screen after the action has completed, you’re left with a focus rectangle that you probably don’t want.

A bad way of dealing with this is to simply use CSS to set outline:none on all A tags. This will certainly solve your problem, but kill all hope of keyboard navigation of your site. Better is to study the way that this interaction happens normally, and then mimic it for your Ajax actions. This is actually fairly easy to do generically:

(function() {

  // Keep track of which link element last had the focus (if browser does not support document.activeElement)
  if (!document.activeElement) {
    $("a").live('focus', function() {
      document.activeElement = this;
    });
  }

  // When any AJAX operation completes, blur any focused link
  $.ajaxSetup({complete:function(xhr, textStatus) {
    if (document.activeElement && document.activeElement.is('a')) document.activeElement.blur();
  }});
}());

Drop the above snippet into your javascript and you should find that focus rectangles still appear and stay visible while the ajax is running, but then vanish once it completes – perfect replica of the effect the user has learned to expect. If your links don’t perform any AJAX, you’ll have to deal with those separately.

Conclusion

With all this to think about, you’d be forgiven for concluding that AJAX navigation simply isn’t worth the bother. But bear with it – the benefits of being able to trivilally maintain state and load small fragments of content really makes a difference, both to the quality of the user experience and the load on your servers.

Try scrolling either of the panels in this example:

You should find it scrolls madly and unpredictably until you press stop.

The first problem is that onscroll is not simply fired when you finish scrolling. It fires like a machine gun continuously while the mouse cursor is moving on the scroll handle. The solution to this is a watchdog. A watchdog is a timer that will execute action A if action B does not happen within X seconds. So every time onscroll fires, we reset the timer, and if it gets to zero we know that the user has finished scrolling (or at least has paused for long enough for us to do something about it).

You’ll find that although it’s not quite as crazed, the panels do keep scrolling, one after the other.

The problem now is that when you scroll DIV 1, and this causes an automatic scroll of DIV 2, that automatic scroll ALSO triggers the onscroll event and so we then scroll DIV 1 again. The solution is to have a variable that flags whether we are currently paying attention to scroll events, and turn it off when we don’t want to detect scrolling (ie just before the auto-scroll) and then on again after the scroll has completed (I’m using an animation library so the scroll takes around half a second to complete). Try this (don’t click the button yet, just scroll the panels):

Success. However, this approach is a bit short-sighted.

You also need to turn off scroll detection when adding or removing content from either DIV, because changing the height of the content within the element can also fire the onscroll event. So if this is a chat window, and we’re adding and removing content in the DIVs, we have to disable scroll detection while we’re doing that and then turn it on again afterwards.

Sadly, our scroll detection flag is crude – it’s a sheer yes/no, and if it’s already set to no (say in order to execute a lengthy scrolling animation), and in the meantime we need to do something quick (say adding a line to one of the DIVs) then that quick operation is going to disable scroll detection (unnecessarily, as it’s already disabled) and then crucially enable it again before the scroll animation has finished. Click the button in the example above to demonstrate this (and then scroll). You should, intermittently, get the unwanted cascade effect you saw in example 2.

What we need is a list, not a flag. Enter the blocker list – an object that collates tokens from each procedure in the script that is currently blocking scroll detection. So if a procedure wants to disable scrolling for a period of time, it adds its token to the blocker list, and then removes its token when it’s done. When a scroll event fires, we now only need to know whether there are any items in the blocker list, and then we can work out if it’s ok to process the event.

It’s also worth noting that there’s no need to actually count the number of items in the blocker list – it’s enough simply to know that there is at least one item in there. And as a further optimisation, we don’t actually need to compute this when scroll events fire (frequently), because the value will only change when some function wants to add or remove its token from the list (less frequent). So we can have enableScrollDetection and disableScrollDetection functions that deal with the blocker list, and which ultimately simply change the old scrolldetection flag to true or false.

There might be a neater way of achieving this, but this certainly works for me. I’d welcome any comments or suggestions. The sources for all these demos are available in the iframes above.