Menu Search
Jump to the content X X
The Smashing Book #5

We use ad-blockers as well, you know. We gotta keep those servers running though. Did you know that we publish useful books and run friendly conferences — crafted for pros like yourself? E.g. new Smashing Book 5 features smart responsive design techniques and patterns.

Why AJAX Isn’t Enough

Advertisement

AJAX calls have moved user interaction on the Web a huge step forward: We no longer need to reload the page in response to each user input. Using AJAX, we can call specific procedures on the server and update the page based on the returned values, giving our applications fast interactivity.

What AJAX calls do not cover are updates from the server, which are needed for the modern real-time and collaborative Web. This need for updates covers use cases ranging from a couple of users collaboratively editing a document to the notification of potentially millions of readers of a news website that a goal has been scored in a World Cup match. Another messaging pattern, in addition to the response request of AJAX, is needed — one that works at any scale. PubSub (as in “publish and subscribe”) is an established messaging pattern that achieves this.

In this article, we’ll look at precisely how PubSub solves the updating problem, and we’ll look at one particular solution (the WAMP protocol1) that integrates both the calling of procedures on the server and PubSub into a single API.

What AJAX Solved Link

Before AJAX, interactivity on web pages was terribly clunky. Any user interaction required an updated version of the page to be generated on the server, sent to the browser and rendered there. In this model, the fundamental unit of interaction was the page. Whatever the browser sent to the server, no matter how small the required update, the result was always a full new page. This was wasteful of both wire traffic and server resources, and it was slow and painful for the user.

AJAX2 broke this up by granularizing things: You could now send data, receive just the result for the interaction triggered by it and then update the relevant parts of the page based on this response. With AJAX, we went from a single generalized call (“Give me a new page”) to multiple interaction-specific calls. With AJAX, we had remote procedure calls3 (RPC) on the server.

Consider the following simple example of a web app for voting made possible by this:

Screenshot of votes demo: banana, chocolate and lemon flavors offered as choices to vote for by pressing a button, plus the possibility to trigger a vote reset.4

What flavor do you like best? (Image: Tavendo1785) (View large version6)

The user can vote for any one of the three ice cream flavors on offer.

Using AJAX, a clicked vote could lead to something like this:

var xhr = new XMLHttpRequest();
xhr.open('get', 'send-vote-data.php');

xhr.onreadystatechange = function() {
   if(xhr.readyState === 4) {
      if(xhr.status === 200) {

      // Update vote count based on call result
      } else{
         alert('Error: '+xhr.status); // An error occurred during the request
      }
   }
}

We would then change just the vote count for the flavor voted for by the user, according to the return of the AJAX call. We’ve gone from rendering an entire page to updating a single DOM element.

This means a lot less for the server to do, and less traffic on the wire. We’re getting a vote count instead of a full page. Most importantly, it enables a speedy update of the interface, dramatically improving the user experience.

What Remains Unsolved Link

In a real-world use case, something like this sample app would have a lot of users voting, often in parallel. Vote counts would change according to users’ combined interactions. Because AJAX calls triggered by a user’s interaction would be the only connection to the server, the user would see the current voting numbers when first loading the app, but they would be unaware of back-end voting changes unless they refreshed the page.

This is because AJAX enables us to update pages only in response to user action on the page. It does not solve the problem of updates coming from the server. It does not offer a way to do what we really need here: to push information from the server to the browser. We need an additional messaging pattern that sends updates to the client without the user (or the client’s code) having to constantly request them.

7

In multi-user applications, distribution of updates is central to functionality. (Image: Tavendo1785) (View large version9)

PubSub: Updates From One To Many Link

An established messaging pattern for handling updates to many clients is PubSub10. Here, a client would declare an interest in a topic (“subscribe”) with a central broker. When the client sends an event for a topic to the broker (“publish”), the broker would distribute this event to all currently connected and subscribed clients.

One big advantage of the PubSub pattern is that publishers and subscribers are decoupled through the broker. A publisher does not need any knowledge of present subscribers to a topic, and subscribers similarly do not need any knowledge of publishers. This means that PubSub is easy to implement in both the publishers and subscribers and it scales well.

Numerous implementations of PubSub are available to choose from, depending on what back-end and front-end frameworks, libraries and languages you are using. For example, for Node.js or Ruby, you might use something like Faye11. If you don’t want to run your own broker, web services such as Pusher12 will host the functionality for you.

Two Messaging Patterns, Two Technologies? Link

It isn’t difficult to find a PubSub technology that suits the needs of a particular app or website. But even for something as simple as our voting demo, we’ve seen that you need both RPC and PubSub — you need to send and request data as well as receive automatic updates. With any of the pure PubSub solutions, you have to use two different technologies for your application’s messaging: AJAX and PubSub.

This clearly has some disadvantages:

  • You need to set up two tech stacks, possibly including two servers, and keep these updated and running.
  • The app needs separate connections for the two messaging patterns, requiring more server resources. These two connections would also both require their own authentication and authorization, increasing implementation complexity and, with this, room for error.
  • On the server, you would need to integrate the two technology stacks into your single application, coordinating between the two.
  • For front-end developers, the concerns are similar: establishing and handling two connections and dealing with two separate APIs.

WAMP: RPC And PubSub Link

The Web Application Messaging Protocol13 (WAMP) solves the disadvantages above by integrating both RPC and PubSub into a single protocol. You have a single library, a single connection and a single API. It will handle all of your application’s messaging between the browser front end and the application back end.

WAMP is an open protocol, and it has an open-source JavaScript implementation (Autobahn|JS2214) that runs both in the browser and in Node.js, allowing you to do pure JavaScript-only applications. Open-source implementations exist for other languages15, so you can use PHP, Java, Python or Erlang as well as JavaScript on the server (and the list of languages is expected to grow).

Diagram of WAMP clients in multiple supported languages connected to a WAMP router.16

With WAMP, you can spread application functionality across multiple languages. (Image: Tavendo1785) (View large version18)

These other languages are not limited to the back end — you can also use the WAMP libraries for native clients, enabling web and native clients to be mixed using the same protocol. The C++ library, for example, is well suited to running WAMP components on resource-limited embedded devices — think sensors in an Internet of Things19 application.

WAMP connections are established not from the browser to the back end, but with a WAMP router, which does the message distribution. It handles the role of broker for PubSub, so that your server just publishes to the router, and this handles distribution of the event to all subscribers. For RPCs, the front end issues the call for a remote procedure to the router, and this forwards it to a back end, which has registered the procedure. It then returns the result from the back end to the caller. This decouples front ends and back ends just like with PubSub. You can spread your functionality across several back-end instances without the front end needing to know of the existence of any of them.

There are additional protocol features on top of the basic routing, such as authentication of clients, authorization based on roles and publication topics, and restriction of publications to particular clients. WAMP routers offer different sets of this advanced functionality.

We will look at how to solve our voting app’s updating problem using WAMP, and we’ll see precisely how WAMP handles RPCs as well.

Live Voting Updates: Vote Using WebSockets And WAMP Link

We will take a closer look at the messaging functionality required by the voting app and go over how to implement this in the browser and on the server. To keep things as simple as possible, the back-end code will also be in JavaScript and will run in a browser tab.

“Back end in the browser” is possible because browser clients can register procedures for remote calling just like any other WAMP client. This means that, persistence and performance considerations aside, browser code is equally capable as code running in, say, Node.js. For our demo browser performance is perfectly sufficient.

The full code for the voting demo is available on GitHub20, including instructions on how to run it and the WAMP router used (Crossbar.io3121). Everything you need to run the demo is free and open-source.

Including a WAMP Library Link

The first thing to do in our code is include a WAMP library. We’ll use Autobahn|JS2214.

For development and testing in the browser, just include it like this:

<script src="https://autobahn.s3.amazonaws.com/autobahnjs/latest/autobahn.min.jgz"></script>;

(This version does not allow for deployment to a production website, and it is limited to downloads from pages hosted on localhost or on a local network IP, such as ones in the 192.168.1.x range.)

Establishing a Connection Link

We now need to establish a connection to the WAMP router:

var connection = new autobahn.Connection({
   url: "ws://example.com/wamprouter",
   realm: "votesapp"
});

The first argument is the URL of the WAMP router. This uses the ws scheme, instead of the http that we are used to, because WAMP uses WebSockets23 as its default transport. WebSockets provides a persistent, bidirectional connection, which enables push from the server without any hacks. Also, no HTTP headers are transferred with each message, which significantly reduces overhead on the wire. WebSockets is supported in all modern browsers. To support older browsers, see “Browser Support24” in Crossbar.io’s documentation.

For the second argument, we need to choose a “realm” that this connection is attached to. Realms create separate routing domains on the router — that is, messages are routed only between connections on the same realm. Here, we’re using a realm specifically for the voting demo.

The connection object we’ve created allows for the attachment of two callbacks, one for when the connection has been established, and one should establishment of the connection fail or should the connection close later on.

The onopen handler below is called upon establishment of the connection, and it receives a session object. We pass this to the main function that we’re calling here and that contains the application’s functionality. The session object is used for the WAMP messaging calls.

connection.onopen = function (session, details) {
	main(session);
};

To get things going, we finally need to trigger the opening of the connection:

connection.open();

Registering and Calling a Procedure Link

The front end will submit votes by calling a procedure on the back end. Let’s first define the function that handles a submitted vote:

var submitVote = function(args) {
   var flavor = args[0];
   votes[flavor] += 1;

   return votes[flavor];
};

All this does is increase the vote count for the ice cream flavor and return this incremented number.

We then register this function with the WAMP router to make it callable:

session.register('com.example.votedemo.vote', submitVote)

When registering it, we assign a unique identifier that is used to call the function. For this, WAMP uses URIs expressed in Java package notation (i.e. starting with the TLD). URIs are practical because they are a well-established pattern and allow the namespace to be easily separated.

That’s it for the registration. The submitVote function can now be called externally by any (authorized) WAMP client connected to the same realm.

Calling the function from our front end is done like this:

session.call('com.example.votedemo.vote',[flavor]).then(onVoteSubmitted)

Here, the return of the submitVote function is passed on to the onVoteSubmitted handler.

Autobahn|JS does this not by using conventional callbacks, but with promises25: session.call immediately returns an object that eventually gets resolved when the call returns, and the handler function is then executed.

For basic use of WAMP and Autobahn|JS, you do not need to know anything about promises. As they’re used above, you can think of them as nothing more than a different notation for callbacks. If you are interested in learning more, however, then HTML5 Rocks’ article26 is a good place to start.

Subscribing and Publishing Updates Link

But what about updating the other clients? After all, that’s what AJAX doesn’t do, and it’s why we’re here in the first place.

To receive updates, a client needs to tell the WAMP router what information it is interested in by subscribing to topics. So, our front end does this:

session.subscribe('com.example.votedemo.on_vote', updateVotes);

We’re just submitting the topic (again, a URI) and a function to execute each time an event for the topic is received.

All that’s left to do is send the voting updates from the server. To do this, we just construct the update object that we want to send and then publish this to the same topic that our browsers subscribe to.

This needs to be a part of the processing of votes. So, let’s add this functionality to the submitVote function that we previously registered, which now looks like this:

var submitVote = function(args, kwargs, details) {
   var flavor = args[0];
   votes[flavor] += 1;

   var res = {
      subject: flavor,
      votes: votes[flavor]
   };

   session.publish('com.example.votedemo.on_vote', [res]);

   return votes[flavor];
};

Well, that’s it: both the submission of votes to the back end and voting updates to all connected browsers, handled by a single protocol. There really isn’t more to basic WAMP usage than this.

Summary Link

WAMP unifies your application messaging — with RPC and PubSub, you should be able to cover everything your application needs. With WebSockets, this is done using a single, bidirectional, low-latency connection to the server, thereby saving server resources, reducing wire traffic and enabling very short round-trip times. Because WAMP is an open protocol and implementations exist for multiple languages, you have a choice of back-end technology, and you can extend your application beyond the Web to native clients.

WAMP makes it easy to write modern, reactive web applications with great user experiences and live updates from the server — and to extend them beyond the Web.

Final Notes Link

(ml, al)

Footnotes Link

  1. 1 http://wamp.ws/
  2. 2 https://developer.mozilla.org/en/docs/AJAX
  3. 3 http://en.wikipedia.org/wiki/Remote_procedure_call
  4. 4 https://www.smashingmagazine.com/wp-content/uploads/2014/11/votes-screenshot-opt.png
  5. 5 http://tavendo.com/
  6. 6 https://www.smashingmagazine.com/wp-content/uploads/2014/11/votes-screenshot-opt.png
  7. 7 https://www.smashingmagazine.com/wp-content/uploads/2014/11/update-server-opt.png
  8. 8 http://tavendo.com/
  9. 9 https://www.smashingmagazine.com/wp-content/uploads/2014/11/update-server-opt.png
  10. 10 http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern
  11. 11 http://faye.jcoglan.com/
  12. 12 http://pusher.com/
  13. 13 http://wamp.ws/
  14. 14 http://autobahn.ws/js
  15. 15 http://wamp.ws/implementations/
  16. 16 https://www.smashingmagazine.com/wp-content/uploads/2014/11/wamp-router-and-clients-opt.png
  17. 17 http://tavendo.com/
  18. 18 https://www.smashingmagazine.com/wp-content/uploads/2014/11/wamp-router-and-clients-opt.png
  19. 19 http://en.wikipedia.org/wiki/Internet_of_Things
  20. 20 https://github.com/tavendo/AutobahnJS/tree/master/doc/_static/code/votes
  21. 21 http://crossbar.io/
  22. 22 http://autobahn.ws/js
  23. 23 https://developer.mozilla.org/en/docs/WebSockets
  24. 24 http://crossbar.io/docs/Browser-Support/
  25. 25 https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
  26. 26 http://www.html5rocks.com/en/tutorials/es6/promises/
  27. 27 http://wamp.ws/why/
  28. 28 http://tavendo.com/blog/post/free-your-code-backends-in-the-browser/
  29. 29 http://tavendo.com/blog/post/websocket-why-what-can-i-use-it/
  30. 30 http://wamp.ws/compared/
  31. 31 http://crossbar.io/
SmashingConf Oxford

Hold on tiger! Thank you for reading the article. Did you know that we also publish printed books and run friendly conferences – crafted for pros like you? Like SmashingConf Oxford, on March 15—16, with smart design patterns and front-end techniques.

↑ Back to top Tweet itShare on Facebook

Alexander came to Web development via getting a law degree in his native Germany and teaching English and German as a foreign language in Dnipropetrovsk, Ukraine for a few years. He appreciates that his current field of work is more logical than the law, but sometimes despairs of always facing a screen instead of other people.

Advertisement
  1. 1

    It’s interesting!

    -3
  2. 2

    WebSockets… Hello?

    19
    • 3

      Yeah… Old stuff…

      -1
      • 4

        WebSockets may be old by Web development time standards – but then that still makes them new technology for so many people out there.
        Anyway, the focus here is on WAMP, which uses WebSocket as _one_ transport, and WAMP (v2) is a new and relatively unknown technology.

        7
  3. 5

    So this is impossible with shared hosting?

    0
    • 6

      Depends on your shared hosting – Crossbar.io needs Python, which may be available, and there are other options for a WAMP router, e.g. Thruway is written in PHP.

      0
  4. 7

    Hi,

    Note that you have a WAMP public test router available here : wss://demo.crossbar.io/

    So you don’t need to install anything to test WAMP.

    I personaliy use WAMP with Python, but you got C#, NodeJS, Java, Erlang, C++ and PHP clients as well.

    If you install the Crossbar.io router on your machine, you’ll get an additional benefit as it also act as an HTTP server and a process manager. It’s quite a powerful beast.

    3
  5. 8

    AngularJS + Firebird = 3 Way DataBind.

    1
    • 9

      In no way arguing against that combination if it fits your use case. With WAMP, the Web is only part of what you can do – it also works across outside of the browser.

      1
    • 10

      Do you mean firebase? Or can you provide an article that explains three way data binding with firebirdsql?

      Anyway, there are better options available than firebase. OT based stuff like sharejs or CRDT based stuff like swarmjs.

      0
  6. 11

    Fantastic article! I enjoyed reading this as I’ve been doing AJAX for quite some time now and have been looking to get familiar with WebSockets and WAMP.

    1
  7. 13

    Ajax is only a transport technology like websockets. You can easily implement a pubsub pattern on top of ajax. Thus pubsub and ajax are not really comparable.

    2
    • 14

      HTTP is just a transport – and AutobahnJS (one of the WAMP libraries) allows you to use HTTP with a long poll solution as a transport instead of WebSockets.
      The point is that WAMP implements PubSub _and_ RPC – and that you don’t have to do any implementation work yourself.

      2
  8. 15

    Good stuff. This is exactly what Meteor is useful for.

    2
  9. 16

    Feeling like socket.io would be a great solution to this problem as well.

    1
    • 17

      Socket.io is well established, and worth a look at for PubSub. What makes WAMP different is that it includes RPC natively. You can, of course, hack RPC on top of PubSub, but I feel it’s best to use a tool that covers my use cases as much as possible.

      0
    • 18

      Couldn’t agree more – almost feels like an omission _not_ mentioning it. Also, there are long-polling type solutions out there if you must go the manual route.

      1
  10. 19

    Thanks for the comprehensive article and introduction to WAMP. It seems like a good starting point for these techniques – now I just need an idea for a cool webapp, rather than just creating ‘non-interactive’ websites ;)

    2
  11. 20

    @Ivan de la Jara

    WAMP uses websocket, but take care of the boiler plate :

    – define a common serialisation format for parameters, return values, and errors;
    – define the routing and behavior for rpc and pub/sub;
    – implementations agree on that and can all talk to each others based on the standard.

    If you do Websocket manually, you’ll have to do all that, and it’s a lot of work, will be less performant, non standard, with no community, no other langage implementation and probably subtle bugs.

    WAMP is nothing magical, it’s a simple convention associated with clean tools that get the job done.

    0
  12. 21

    If you found this interesting, take a look at meteor.js. If it fits your project, it can save you some time!

    0
    • 22

      “If it fits in your project” – then use Meteor, sure. Always use the best tool for the job. WAMP is a different fit than Meteor: it’s a library, not a framework, and focuses only on messaging. With the routed RPCs (i.e. the endpoint offering the procedure can be anywhere, without the caller needing to have any knowledge about this) it has a unique feature. And the fact that implementations exist for multiple languages, it extends beyond the browser.

      0
  13. 23

    Great article! Very informative, I enjoyed going through it.

    0
  14. 24

    Just a caution, anyone who is susceptible to DDOS (which is basically everyone now) needs to be very, very careful with these technologies. Its very hard to filter them out, so while services like CloudFlare are just starting to be able to support websocket type architectures, it’ll be expensive for you.

    I’ve been building web sites and apps for nearly two decades and have had no problems. In the past year we got hit three different times, on three different projects, for no apparent reason. The tools for DDOS are so cheap and available now that its very easy to become a target. Cloudflare is great if you are running a traditional or even an AJAX site, but until protection catches up, use this article’s tech with caution or have a well funded infrastructure to protect against attacks.

    This is something I wish someone would have told me a year ago… nothing more frustrating than not being able to SSH into your server and not knowing who’s hitting you, let me tell you.

    0
    • 25

      *Edit – when I say “hard to filter them out” I meant “hard to filter websocket traffic out of DDOS traffic”

      0
  15. 26

    Those in the .net world should check out SignalR (http://signalr.net/) – which does pub/sub quite elegantly and works with the most appropriate transport available for the client’s browser (server sent events, AJAX long polling, web sockets, etc).

    0
  16. 27

    This it’s very like the MS .Net SignalR… There’s some security issue about this technology?

    0
  17. 28

    Thanks for writing this, I found it very interesting. Many years ago, I built a site as a proof-of-concept for this kind of thing. It’s a web-based virtual world that keeps all players in synch … essentially a graphical chat environment. It used frame-based AJAX and each client pinged the server every 3 seconds to get new remote updates, and immediately when a client had local updates to share. It worked surprisingly well … even though it was built in 1998, 6 years before AJAX emerged as common practice. It’s still up and still works … http://worldwideworlds.com

    1
  18. 30

    Shouldn’t one of the code snippets be as follows:

    connection.onopen = function (session, details) { // first argument was “sess” which is an error
    main(session);
    };

    0
  19. 32

    Learned something new from an old subject. I love the level of details in this article. Thanks!

    -1
  20. 33

    Not to break your bubble, but…

    Ajax web sites suck in 90% of cases. Twitter. JIRA. Some over-the-top Tumblr templates.

    I hate using them. They suck. They make my browser crash. They are buggy. They take forever to load things a standard app would take microseconds.

    So yeah. This happens when tech people and marketing people forget about real world use cases.

    -8
  21. 35

    I’m going to go along with Dave Sanders here on the DDOS aspect, and even extend a counter-argument. The web is based on REST. AJAX, while not precisely RESTful, at least doesn’t break the call-and-response model. PubSub direct to browsers creates the possibility of unsolicited packets being sent into a non-very-robust run-time environment, namely, a web browser. DDOS attacks aside (and I’m not downplaying that aspect), once a given PubSub environment becomes sufficiently populated, it opens the door for port scanners to detect instances, and then attempt arbitrary-code execution attacks on those browsers. Browsers can be compromised by specially-designed web pages already, and that’s why tools exist to warn consumers against going to known dangerous websites. Now we should open vulnerable browsers to drive-by code execution attacks?

    I don’t want extra listening sockets on my systems. (This is why I have a firewall, right?) It’s a fundamental principle of IT security that connectivity needs to be initiated on the client side, not on the server side. One notes that the FAQ on wamp.ws fails to include the question “is it secure?”

    Honestly, polling works. If you want real-time updates, maybe browser-based PubSub solutions aren’t the way to go. (It won’t even be real-time anyway since a given broker is still limited by latency and a fixed number of available sockets.) A case might be made for in-house applications that can’t be accessed from the broad Internet–but outside the firewall? No way. Haven’t we learned our security lessons yet?

    4
    • 36

      I’m not an expert on security, but a few comments about WebSocket security as I understand it:
      The connection is initiated by the client via a HTTP get which contains an upgrade request to a WebSocket connection. The listening port here is the server, which needs to listen for HTTP connections anyway.
      After the upgrade, the connection is just between the client and the server. This can be secured via TLS.
      There are no listening ports in the browser that an attacker could scan.
      As to polling – of course this works, up to a point. So did a lot of other technologies throughout the history of the Web. That’s no reason not to use newer technologies which are better suited for a particular task. For the benefits regarding wire traffic of WebSocket vs. polling, take a look at http://www.websocket.org/quantum.html.

      3
  22. 37

    I’ll agree with @Brook: for everything I’ve worked on, a periodic AJAX poll can replace server-originated updates enough that I don’t really feel the need to invest in a whole new stack just for SubPub et al. Heck, for most of the apps I’m working on, I even think AJAX is often overrated: if you design your interfaces well and think through caching properly, static pages really aren’t the end of the world.

    1
    • 38

      As you said: It all depends on the use case. I’m not advocating WAMP as fitting for everything – nobody should do that with any one particular technology.
      Having said that, I’ve found WAMP useful in implementing quick, responsive interfaces, especially if you want to sync state between instances.

      0
  23. 39

    No mention to Server Sent Events ? It is the most appropriate underlying technology to get events from the server, better than WebSockets regarding browser support.

    1
  24. 40

    Interesting read. Many apps still use ajax but I guess it will be replaced by javascript. My site http://instantdomainsearch.net uses ajax and it does more than enough for me for the time being.

    -2
  25. 41

    WAMP — not to be confused with the WindowsApacheMySqlPhp stack! Great article on the topic. I remember the first time I built a Node.js live chat application using WebSockets, it was a really eye-opening experience in terms of new possibilities. It will be exciting to see the ways wider-spread adoption of these techniques will affect web applications in the years to come. I’d image we continue to see some pretty unique and ingenious implementations that tend to catch on and ultimately provide better UX in general. (Which is awesome, because that should be the main focus anyway!)

    -1
  26. 42
  27. 43

    That’s why you should try Meteor. It is the future of web development. It has everything you need and will make you a faster and better developer. Just give it a try and you will not regret.

    -1

↑ Back to top