At an event a couple of years ago, I made the mistake of claiming to an elearning manager that people would one day search for information through social networks rather than flat text search engines like Google. That was pretty much the end of the conversation; he couldn’t believe that this would ever be the case.

Interesting, then, to see Robert Scoble claim that Google will be beaten by the likes of Facebook and Mahalo. In other words, graph based search that provides results based on your interests and relevant connections.

In the linked article, Michael Arrington points out the genuine flaws in some of Scoble’s claims, but I think the two search approaches essentially augment each other. The first, traditional set of search, allows for broad information discovery: for example, to find out when the local DIY superstore shuts on a Sunday, or to find alternatives to my broken digital TV service (as I did yesterday). The second allows you to build up a network of trusted sources and mine them for information you trust.

A static example of this current approach is Google News, which I use daily to check out different takes on current events. (The CNN version of a story often has information that the Guardian doesn’t, and vice versa.) News sites are, in effect, a subgroup of sources that I might want to mine for information. But what about people in the tech sphere, if I want to find out about approaches to programming a particular kind of project? Or how about finding information relevant to my business from my colleagues? (Or taken from the global search and edited for relevance by their actions?)

These types of search are underdeveloped and underused, in part because everyone wants to ape Google. As Arrington points out, the next big thing may be from Google itself; we just need someone to think outside the box and push the concept forward from its current, stale incarnation.

This just goes to show: you can add all the features and complexity you want, but they won’t necessarily make the user experience better.

So, 24 hours without Skype access later, and I still can’t get on (although apparently some can).

This has been an important lesson in single point of failure (which is one of the core reasons I’m always harping on about decentralised services). We use Skype pretty much universally to talk to each other; although alternatives exist, it’s the one that almost everybody is almost guaranteed to be on. The alternative is the traditional phone network, which is nowhere near as cost effective.

It’s forced us to create a new communication plan, and I’m certain we’re not the only tech company in the same boat. The Internet has meant that we can resource people from all over the world; it’s no good if we then can’t talk to them.

Annoyances aside, my thoughts go out to the Skype team, who probably haven’t slept at all in the last day. They’re going to be under serious pressure from both their millions of users and their parent company; I hope they work it out and can get some rest soon.

Update: here’s a plausible explanation for the mess.

To everything, at least.

OpenID is a fantastic little protocol that lets a user log in with their username from another service. It means you can log into Explode with your AIM screen name; you can also log into Livejournal with your Explode profile URL. The OpenID client site shoots the user’s web browser over to the OpenID server site, authentication is performed, a token is passed back to the client, and bob’s your uncle.

As far as this goes, it’s simple, powerful, and very clever. If you’re building a new web application, I highly recommend including OpenID functionality – even if you don’t switch it on for everybody. But what happens if you want to do something behind the scenes?

The larger web applications are typically built around a number of server architectures, which each perform a different task. One might process some data on the back end; another might be reserved for displaying thumbnail images. If all the servers are owned by one body, they can each be passed a token (maybe even a cookie if they’re all subdomains of the same parent domain), and poll the authentication server for the current user’s details.

Now imagine you want to build a decentralised version of that using web services which are all owned by different organisations – but web services that need to know about who you are. One might provide storage, another might provide a profile, a third might be a messaging application. How do you provide a generalised way of passing authentication across, if there’s no central authentication authority (as there isn’t in OpenID, or in a decentralised system), and you don’t want to go through the user’s web browser each time?

Don’t say Shibboleth.

None of the standards out there match the simplicity of OpenID, and therefore stand a chance of being as widely adopted. SAML, nominally the standard, requires SOAP, which brings its own problems. Even OpenID, when you examine the number of supported services, is very far away from becoming a mainstream standard. Federated identity is tough, but I think part of that may be to do with the way these standards are created; often they’re the products of committee development, either in an institutional or corporate field. In both cases, the demands placed on the spec by the various stakeholders are inevitably going to cause bloat and inefficiency; the two surefire things that will prevent standard adoption. OpenID, meanwhile, has a very small spec, does what it’s supposed to and nothing more, and even has pre-written code classes available via JanRain.

This is one of the issues we’ll be bringing up at the Data Sharing Summit next month; I’d be interested to hear your ideas. Project Higgins looks to have a very healthy (user-centric, protocol-agnostic) attitude towards identity federation, and they may be one to watch.

Dave Winer gushes about Facebook adding a handful of RSS feeds. I’m less enthralled, which is underlined by some of the quotes from commenters he points out:

Jeff Sandquist says, “I suspect this will allow me to send my Facebook status updates to Twitter.”

Seriously, I think I just threw up a bit in my mouth. Surely there are more interesting applications we can find for web application interoperability than Facebook to Twitter? We have the potential to build a far-reaching, global, open platform that is available to anyone with a connected device. People are building networks and clouds and new devices we won’t have even thought of yet; RSS on Facebook seems like just a bit of an anachronism.

Misja has posted about the Elgg language contributions, which are impressive: 49 languages and counting! A big thank you goes to the Elgg open source community.

I’ve also been remiss in not giving him a full welcome: Misja Hoebe was the latest addition to the full-time Curverider staff. We’ve been collaborating for a number of years now, having originally met through this community, and we’re really pleased to have him fully on board.

When Dave and I set up Curverider a couple of years ago, the response was lukewarm. Elgg, as an open source product, was supposed to be free, wasn’t it? What was a commercial company doing with it? Did that mean it wasn’t going to be open source any more? Rumours flew round the e-learning blogosphere faster than we could stamp them out.

Today, we announced that Curverider has attracted investment (here’s Dave’s comment), and I’m kind of expecting the same thing to happen. In fact, this is going to herald a new chapter in Elgg development; a new chapter that is still resolutely open source, licensed under the GPL, and committed to open standards.

Curverider provides commercial support, as well as customisation and consultancy services. Through a partner arrangement, we also provide hosting for your Elgg environment. Because of the way this works, Elgg is free: it’s always been available to the community, and it always will be. However, the money from related services allows us to work full time on it, and even hire new programmers to join our team.

We’re a rare breed: a British web startup that’s received investment. One that deals with open source and free, open standards is rare even in Silicon Valley. With this announcement, we’re not only here to stay (we were sustainable anyway), but you can expect us to become bolder and more ambitious. Watch this space.

Over on the DataSharingSummit board on Facebook, which I ironically can’t show you because Facebook doesn’t allow for public data sharing, Marc Canter asked what rights we think end users should have. I replied as follows:

I think there’s a serious danger of overthinking the discussion. If we wanted to, there’s enough meat in this topic to keep us going for decades; surely the summit and surrounding discussions should be more productive than that?

From our perspective, which is heavily influenced by education and non profits, we’ve been talking about two main issues:

1. My freedom to take my data somewhere else at any time

2. My right to stop you from taking my data somewhere else at any time

Here in the UK, the second is particularly important if we’re dealing with networks that will be used with schools etc. The Data Protection Act has certain requirements that, currently, a lot of people disregard. (In addition to the usual issues of whether or not Myspace really makes a great teaching environment … which is a discussion for somewhere else.)

Commercial considerations affect both sets of rights; for example, I think it wouldn’t be a piece of cake to export this discussion somewhere other than Facebook, except by copying and pasting. At the same time, there’s very little to stop Facebook from highlighting this thread in a publicity document or selling data about DataSharingSummit participants.

I think a user should have full ownership over his or her data, and that social networks will inevitably evolve into a cloud of features and services that don’t behave in the walled-garden way we’re seeing now. In the first instance that means there should be generic, standardised ways to import and export data; in the second it means there need to be ways to *find* it across different services. (One bonus that a walled garden site like Facebook has is that you know all the data can be found with the Facebook search engine. Once people start moving it all over the place, the community becomes fuzzier.)

The right to not have your data exported is tricky, because that’s how a lot of networks make their money. I know of at least one that’s making not a small amount farming its data out to a marketing analysis firm, for example, which pays it handsomely for use as a test case.

Perhaps that’s a big question that needs answering: how do we make interoperability worth it for the existing networks, for whom the data they currently sit on is probably their main asset? Or are we confident enough that the smaller networks and network software are going to put enough pressure on them by themselves? Establishing these rights and standards isn’t going to mean much if 80% of user accounts don’t support them, except perhaps for the principle of the thing.

On the other hand, maybe that won’t matter as we move from social networks to collections of software that have social features.