XMPP: powering the real-time, really live web

June 12, 2009 | 7 comments

When Google Wave was previewed last week, most people focused on its Gmail-like user interface and the slick way it handled collaborative spaces in real time. What was more exciting from a web technology point of view, although much less sexy, was the underlying protocol: Google Wave is built on top of an extended version of an open source real-time data standard called XMPP (aka Jabber when it’s used as an instant messaging context). This means that anyone can build servers and end-user applications that are compatible with Wave.

Yesterday, the BBC announced that they were using an open source framework called Hemlock in order to provide real-time web games on their Childrens’ site. Hemlock’s interfaces are built in Flash, but the real-time aspect is powered by XMPP. Developer Ron DeVera explained why over on the Hemlock blog:

XMPP is commonly used to enable chatrooms, like Gtalk or Facebook Chat. However, instead of just sending text, Hemlock lets you send all kinds of data at the speed of instant messaging. Even better, XMPP is designed to scale up and send data to many people at once, so you can push that data to multiple people in real-time.

In other words, it’s perfect for real-time applications on the web.

Traditionally, live web applications have used a development methodology called AJAX, which stands for Asynchronous Javascript And XML. In order to provide the illusion of a real-time application, the web browser needs to continue to ask the server if there’s any new data to ask, via a new AJAX request – which is a waste of bandwidth and resources both on the user and server side. XMPP, on the other hand, pings the client software whenever there’s new data.

Servers already need software to host and process dynamic web pages, as well as a database server to handle most types of user data. To use XMPP, you need to add an XMPP server to the mix – something that in the past was more fiddly than most people were willing to play with. However, times have changed, and ejabberd is free, open source and easy to install on Windows, Mac OS X and Linux servers. (I’m still at the early stages of playing with it; I’ll let you know how I get on.)

The XMPP community is still very small compared to the wider web, but developments like Wave and Hemlock should accelerate growth and interest in it. There are some interesting projects springing up, and lots of libraries available for XMPP development; XMPP programming is a skill developers should get in on now, before it really heats up.

Related entries

7 Comments

  1. Ben, you are right on about XMPP being a key tool for future Internet apps. More and more social networks are going to start combining mobile phones, websites and real time interfaces like Skype.

    I’ve been experimenting with it on and off for years (ever since Google Talk was introduced at least and probably before). The main problem has been convincing clients that they should look at something beyond a bog standard website.

    I hope that Wave will be more successful in sparking the long delayed XMPP revolution than jabber (or Google Talk) has been.

    Kevin Jardine June 12, 2009 (5:56 pm)
  2. The big problem with XMPP for me is that for lots of small messages going to different places (the typical usecase) the overhead is rather high.

    That’s not to say it’s not cool, but it is a little bit of a problem if you’re talking widespread deployment and scaling.

    I’m wondering if – given that CPU is cheap compared to bandwidth – some sort of symmetric mod_deflate could be a simple solution (if that hasn’t been done already), although that would require decompression of all packets that need to be analysed which may have its own inefficiencies.

    How much of this is actually a problem given the current bandwidth explosion remains to be seen, but spare a thought for those in places like Russia and the far east (or on mobile platforms) who are paying for connectivity per megabyte.

    Marcus Povey June 15, 2009 (9:53 pm)
  3. nice words. liked to read it. In my oppinion xmpp is missing a real flagship of instant messaging service or product. who uses GTalk today?

    two years ago I started to get into xmpp topic and put all my hope into Jive Software with the Openfire (formerly Wildfire) Server. Great product but today the project stands still. http://www.igniterealtime.org

    Now I hope the best for Google Wave although I am not the biggest fan of Google. But if Google’s “Wave” becomes succesful I think nobody will know about xmpp better than today. In the demo from Google there was no talk about xmpp at all. It’s Google’s Protocol.

    Let’s hope there will be a quick improvement. We currently use Openfire in bussines but without further updates an bugfixes there will be no chance of keeping it online.

    cheers

    peter June 17, 2009 (8:34 pm)
  4. @Marcus XMPP already does include compression, in two flavors: compression at the transport layer via Transport Layer Security (TLS) and compression at the XMPP application layer using zlib or other methods via Stream Compression (see XEP-0138).

    stpeter June 17, 2009 (9:39 pm)
  5. Nice article Ben. XMPP has its overheads and a few downsides but overall its a very elegant protocol that can be built upon. There are a large set of XEPs that extend it. I love the hemlock idea. XMPP is designed to be easy on the client and complex on the server. But for widespread use, it needs to be super easy like HTTP and also it needs tool support.

    Ravikant June 20, 2009 (9:41 pm)
  6. @stpeter I kinda figured someone would have thought of this already ;)

    Marcus Povey June 24, 2009 (2:14 pm)
  7. http://devcentral.f5.com/weblogs/macvittie/archive/2009/06/02/facepalm-google-wave-choice-of-xmpp-not-the-death-of.aspx

    “About XMPP usage in Google Wave–it’s for server-to-server communication, while HTTP is still fully employed for the usual server-to-client communication.”

    So does these arguments and examples in that article mean in the case of Google Wave, *real-time* between the client and server is still guaranteed by XMLHttpRequest, not XMPP? Do you agree with him?

    dahuang June 28, 2009 (5:07 pm)

Leave a comment