Making billions of dollars from the federated social web

August 28, 2012 | 4 comments

Diaspora was pretty exciting. A Kickstarter campaign that promised it all: a platform created “for everyone to have full control over their data and to empower people in to become responsible, secure, and social Internet dwellers”. They raised over $200,000, which at the time was the largest Kickstarter funding round ever. Neat!

Yesterday, Diaspora announced that they were becoming a community project. That means that the wider open source community will be responsible for its direction; the original team will be involved to some extent, but are now involved in projects like, which just made its way through Y Combinator. ( is released under the same open source license as Diaspora: here’s the GitHub repository.)

The federated social web is a holy grail for many of us. It’s a simple idea: rather than ceding control of our data and activities to a monolithic, monopolistic third party, why don’t we create an open market for services we can use, which can all interoperate? Those of us willing to set up our own servers would own our own data; others would use the servers set up by their schools or employers; others still would join the hosted service of their choice. The result would be a more innovative, web-like application web, where niche interests could be better served and people could choose the applications and interfaces that best suited them.

It goes far beyond Facebook-style social sites: the potential is for deep interoperability between every application on the web and beyond – and, potentially, identity spaces where we could keep our personal information safe.

Back in 2005, I tried to make the case in a letter to the social web hating journalist Andrew Orlowski:

If a thousand sites depend on Flickr, what happens if Flickr goes down? Wouldn’t it make a great deal more sense to think about standards for data transfer and availability to allow for Flickr-like data all over the Internet, rather than in one place – effectively a peer-to-peer network (or networks) of tagged resources? That way one can still grab information and manipulate it, but with zero dependence and a high tolerance for network failure. Rather than there being one nebulous Microsoft Office 12 For Web, there could be ten thousand office applications that all use the same standards and allow for data transfer perhaps with storage services elsewhere. The answer to “how bad do you think it will get” is “exactly the same as now” if we allow one service provider to create a monopoly for a particular type of service. As ever, the solution is open standards with a framework that anyone can use. This is obviously not going to lead to billions of dollars for any one provider, so it’s probably up to the open source movement to create.

Returning to that letter seven years later, I think I was wrong about one thing: I believe it is possible to make billions of dollars from interoperable social web applications – and there’s no need for mass public adoption of apps using federated social web standards to do it.

I’m convinced that business software should be using federation protocols, as long as those protocols have built-in access permissions. Imagine if your organization – a government department, say, or a corporation – could selectively create shared spaces for joint projects with other organizations, for as long as the project existed. Imagine if you could collaborate in more fluid ways, where applications were more like documents that you could introduce into a project when you needed them. Imagine if email was smarter. Whereas decentralization in public consumer apps is an ideological feature, in enterprise apps it has a solid business case and adds measurable value. Businesses will become more profitable by using federated apps.

On hearing the news about Diaspora, StatusNet’s Evan Prodromou made a very sensible plea:

I only have one favour left to ask: please, for the rest of us who are still working full-steam on federated social networks, don’t fall into the comforting fiction that the problem is insurmountable. It’s not; it deserves our attention and support. You didn’t waste the last two years on something pointless and unattainable. Your work matters.

It might sound like hyperbole, but I still believe it could change the way we do business, and ultimately how we all communicate.

Update: In the comments, Johannes Ernst points to his post about why decentralized software is 10x harder. I think he’s right about the problems that need to be overcome. But check out his note at the bottom: watch this space.

Off the hooks: user-centered development in a mobile-first world

June 11, 2012 | 1 comment

The following post contains my notes for a talk I gave at Over the Air 2012 at Bletchley Park on June 2nd, 2012. They build on my notes for a previous talk, Building the user-centered web.

Desktop isn’t enough.

For the last few years, I’ve been working on latakoo, a service that lets you share video fast using any Internet connection. Video is, traditionally, a pain to upload, due to very large file sizes, combined with the limited upload bandwidth inherent to most commercial connections. We solve this by compressing the video first, using a very simple interface: just drag, drop, click, and your video is uploading, and shared in a private, enterprise environment. There’s no need for any substantial training or technical video knowledge. It just works.

We support a lot of different video formats, and the resulting file is often 1-4% of its original size, while remaining very high quality. A lot of TV stations use latakoo – the Science Channel used us to send footage of the SpaceX launch back to base – but we also have integrations with Facebook, YouTube, Box with more to come. latakoo’s a useful service, and a great team that I’m proud to be part of.

However, an app that runs on a desktop or laptop computer isn’t enough. Sure, the latakoo app works great in those environments. But a lot of people began asking us if we could make it work on mobile – including those TV stations. TV news stations are arming their reporters with iPhones and sending them out in the field instead of laptops.

Video compression is no problem for my i7 MacBook Pro – but can it work on my handset?

Of course it can.

latakoo iPhone appMobile devices are this century’s personal computer.

I have latakoo apps on my Samsung Galaxy S2 (in beta, coming soon), my iPod and my iPad. All of them are very comfortable compressing video – because all of them are relatively powerful mini computers.

My Samsung Galaxy S2 has a declared clock speed of 1.2GHz and has 1GB of RAM; the Galaxy S3 has four cores at 1.4GHz, and 2GB of RAM. It wasn’t so long ago that laptops were being sold with those specs, and it’s no accident that mobile growth is outstripping the rest of the computer industry:

Oh, and here’s one more quote:

“We do not currently directly generate any meaningful revenue from the use of [our] mobile products, and our ability to do so successfully is unproven.”

That’s from Facebook’s IPO filing. Over 50% of Facebook users access the service through a mobile app or site – yet, until the last couple of weeks, they had no way of obtaining revenue from these accesses. (Their promoted posts feature is transparently a way of obtaining revenue in a way that applies to mobile users.)

As we know from Building the user-centered web, the web has transformed from a series of context-free, stateless pages to interlinked applications that have social functionality at their core.

Curiously, despite this social focus, each application is its own atomic destination with its own URL, and is by default only aware of data created within it. That means we need to register for each application we want to use, fragmenting our accounts over potentially hundreds of products and company data centers, and that the documents, files and data we create within them can’t easily be shared with other applications. Currently, each web application is like a silo: they exist on their own, and if they interoperate at all, it’s through specific links between applications that have to be individually developed.

Mobile-first social apps exist, and are beautiful – but are being designed to this silo model. Path, Instagram, Viddy and Socialcam are four examples of gorgeous apps that are designed to trap usage in their own databases, and often spam users into joining. After all, the number of users in your database, and growth thereof, is the predominant measure of value in Silicon Valley. Instagram certainly wasn’t worth $1 billion to Facebook for their revenue.

SilosSilos stifle collaboration.

Collaborative spaces in social web applications are like documents: they’re one of the currencies of the social web. Just as I need to be able to use my word processor of choice to edit a document, I need to be able to use my social tool of choice to collaborate with others – and if I want to share a collaborative document with someone, I should be able to do so regardless of the social service they happen to use. Where I choose to host my identity should not dictate the kinds of content I can create. Not to mention that some companies simply can’t use existing social platforms, for ethical and/or legal reasons. The UK’s Data Protection Act, for example, precludes the use of Facebook in schools.

Right now, we have to register with each application we want to use. What if we required each application we used to register with us, in digital identities under our own control? What if, using these identities, anyone could connect to anyone else, and anyone could store their data anywhere as long as the storage provider followed the same broad standards?

The web would become truly social.

Your ability to collaborate is not subject to a single company’s success: social functionality and application infrastructure are inherent in the network itself. The possibilities for collaboration are not subject to technology beyond common open standards, which can evolve. A wider range of application possibilities is ensured, because web applications gain the ability to interoperate in a general way. Finally, privacy and user control are established by allowing a person to determine which application has access to which data.

Furthermore, when you remove the silo, private or short-term collaborative spaces become possible:

  • Transience: there’s no need for the community to exist for longer than it has to.
  • Simplicity: there’s no effort involved. Once you’re done with a community, you simply close the communication.
  • Privacy: it’s very hard to share activity with the wrong people.
  • Decentralization: the community is physically hosted between all the involved parties, removing single points of failure.

This user-centered future is being built.

One of the next big hopes is Web Intents. Here, the user performs a generic action in an application, like “share”, perhaps by pressing a button. The user can then choose from all the services he or she prefers, that support this particular action in this context. The list is built automatically from the user’s preferences – a set of applications which are, indeed, registering with the user’s central identity.

Colour coordinated appsThis user-centered future has been built – on mobile.

Web intents are, of course, an adaptation of Android’s intents model. (Albeit in a simplified form.) And, in fact, we already do all kinds of identity-like things with our mobile devices.

  • We can pay with mobile.
  • We can authenticate with mobile (as many of us do with online banking).
  • We organize our contacts, our calendars, our messages, our photos and our videos on mobile. We speak to our loved ones on mobile. We read and watch and play on mobile. We write the identifying address of our mobile devices on our business cards, and have done for years.
  • Mobile knows your context: where you are, and who you’re with, where you’ve been, and where you plan to go. It’s far smarter, and far more social.
  • Mobile is already a decentralized social application platform, with a wide range of communications methods, from SMS through Bluetooth and NFC. We can use all of these methods, depending on the context, to send social information.

Smartphones already let us share in a decentralized way, but only specific, agreed data types like SMS, MMS and email. So, how can we expand on this to send arbitrary social data?

Sending social data on mobile devices

It turns out that the Activity Streams standard allows us to handle social data in a generic way, that can carry specialized data for niche application types. For example, a task in a distributed enterprise system might be an extention of note, one of the base ActivityStreams object types.

ActivityStreams is an extension of Atom, so it could contain the content itself – although this might not be feasible on mobile. We could send mobile notifications with ActivityStreams-like payloads (up to around 1400 characters on iOS, or around 1024 bytes on Android). If an application handles a particular object type, great! Otherwise, perhaps this is a great opportunity for the platform provider to gently point to their app store to make some suggestions.

This could be a solution for direct notifications, but still need to read updates from a central list.

Abstracting from individual mobile devices

I carry two phones: one for the US, one for the UK. I also carry an iPod for when I have to use an iOS app, which has its own set of profile data, although fewer communications options. And indeed, because of contracts and other factors, people upgrade their phones every 18 months to two years, possibly changing their numbers. Handsets get stolen, and some people have entire bags of SIM cards that they use on their travels as appropriate. And finally, backups are important.

An abstraction service – like a social version of Google Voice – may help here, gathering messages in a feed and sending notifications to handsets, or allowing handsets to retrieve as a batch.

Once you’ve done this, the obvious question is: could this model work on the desktop or in a pure-web model? Sure, but by thinking mobile-first, we get to build on work that’s already been done on these platforms. Mobile platforms lose a lot of the legacy baggage inherent in both desktop operating systems and the web, and they’ve been designed from the ground up to support distributed communications in an intelligent way. This doesn’t diminish the importance of the web as a cross-platform home for both applications and data, but it provides new opportunities for linking people data and building next-generation social applications.

Everybody benefits.

Platform owners get to build more engagement – and therefore value – into the platform itself.

Application owners get new ways to spread their software – through organic social proof.

Users get freedom from the silos that stifle their possibilities for collaboration and creativity.

Some questions:

  • A lot of people are still on Symbian and other legacy mobile operating systems (notably all those Indian users). How can we reach them?
  • How could we prevent spam and ensure data privacy on such a platform?
  • Which would be the best platform to start on? I think Android technically has the most support for this model, but Windows Phone may have the most organizational support.
  • In an era where platform value is so closely tied to trapped engagement, would anyone fund this?

You need to be wearing Google goggles to think this is a good idea

April 4, 2012 | 1 comment

Today, Google released this video of Google Glass, an augmented reality project:

Beautiful, right? It’s a virtual assistant that sticks with you wherever you go.

Exciting features debuted in the video include:

  • Automatic geo-tracking of both you and your friends.
  • Seamless photo-taking and live video broadcasting from anywhere you can put your head.
  • A head-up display that occupies some of the visual channel.
  • Everything you see – your visual reality – is augmented through a single company.
  • You are continuously recording information about the people around you as well as yourself.

Okay, I confess: by beautiful, I meant scary. Leaving aside the implications of having our ambient information spoon-fed to us rather than discerned through inference and discovery, this project opens an ethical can of worms, and asks important questions about Silicon Valley’s attitude towards other peoples’ privacy.

Indeed, traditionally, this isn’t a can of worms that Silicon Valley has been very good at dealing with. Android phones encourage you to link your friends’ accounts together, joining their Facebook details to their Google details, for example. There hasn’t been a strong line between information that is yours, information about you that you’re happy to share, and information that you’re gathering about other people. It seems clear to me that people should know what they’re sharing about themselves, should have to opt in to do so, and should not be able to share personal information about other people without those people opting in. That doesn’t seem to be an opinion that Silicon Valley shares with me. Most free services are a Katamari-like information-gathering free-for-all.

I’m a fan of many, many things that Google does, and kudos to the product team that’s putting Google Glass together. It’s a very significant achievement. But from a social perspective, there’s a lot of work still to be done.

Identity is the operating system

January 15, 2012 | Leave a comment

"Dude, you can make calls on your camera?!" (photo by @troy)I’ve got a phone number: +1 (312) 488-9373. Feel free to call or text it.

If I’m walking around, you’ll get me on my Samsung Galaxy S II. If I’m in transit (but not driving), you’ll probably get me on my iPad. If I’m at my desk, I’ll answer and take the whole call through my laptop. For you, the experience of contacting me will be exactly the same (give or take some background noise). For me, the experience fits my context: I can make and receive calls and texts on any of my devices. The same is true for email.

Consumption works the same way. All my important files are stored on Dropbox. If I need to get at something – for example, a work-in-progress piece of writing, or a receipt – I can pick up any of my Internet-connected devices and grab the contents. Similarly, my notes, which I take through Evernote: I can create and consume these anywhere.

A final example: I love movies. Watching them at the theater is still magical for me, but I also enjoy them elsewhere, depending on what kind of movie it is. (My favorite streamable new release right now is Midnight in Paris. A great film.)

At home, I use my dedicated media PC to play through a sound system and flat-screen monitor. It’s not perfect, but it works. Elsewhere, I might use my laptop or my iPad. On the road, I can play the same movie through the same service on my phone, or I may be able to take a downloaded version for offline consumption.

So far, so obvious. These are all known use cases that demonstrate why the consumer Internet is so powerful. But I have a question:

Shouldn’t applications, services and content be sold to me, instead of my devices?

Right now, I have to set each of my services up on each of my devices, and tell them to use the same account. That sometimes doesn’t work perfectly: for some reason, for example, I seem to have two Path accounts – one for my US handset, and one for my UK handset. I’m not sure how this happened.

Ideally, I want to sign up:

  1. Once for each device, to tell them that I own it, and where I store my identity.
  2. Once more for each service or application, to associate them with my identity.
  3. The filesystem would be networked and bound to the identity. So rather than storing it on its own infrastructure, Evernote would save notes to my filesystem, which could potentially be accessed by other networked software.
  4. Each identity would have an Internet-accessible unique identifier and point of entry.

The applications would then automagically become available on each of my devices. Crucially, when I go to buy or rent Midnight in Paris, it then is also available on all of my devices, because I’m renting via my identity rather than any one device. For the next 24 hours, say, I can stream the movie wherever is most convenient. If I buy a license for Microsoft Office, then it is licensed to my identity and I can use it on any of my devices.

This is literally a per-seat model for selling software. It makes buying and consuming simple, and will reduce piracy.

So here’s a follow-on question. It’s a dull-sounding one, but bear with me:

Wouldn’t this make enterprise provisioning dramatically easier?

Right now, system administrators in enterprise environments push software over their networks, and often refuse to allow non-approved hardware onto their infrastructure in order to make this easier. In an identity-centric model, though, where applications are delivered using Internet technologies:

  1. Software would be provisioned to identities rather than machines.
  2. The available identity domains and software on any given network could be locked down as appropriate (so, for example, I could bring in my smartphone but only use a sanctioned identity with it if I wanted to connect to the local network).

Wouldn’t this make consumer applications dramatically less annoying?

For me, the answer is “yes”. I don’t want to care about my devices and their capabilities, and I’m a CTO with a computer science degree. End users want their software to seamlessly “just work”, and they want to seamlessly be able to move content from one machine to another, or share to another person no matter what that thing happens to be, or where their data is stored. Right now, operating systems have become gatekeepers: bottlenecks that get in the way of users.

For me, this is the real application of a decentralized social web. It’s not just about sending messages around – it’s about using the Internet to create a fabric of interdependent applications where we retain control of our data (those identities and networked filesystems could be anywhere) while enjoying a simpler experience. Application providers and content owners sell more of their products, because they’re easier to consume, everyone loves their devices that little bit more, and every new product sold becomes a window onto a much bigger, connected ecosystem that is greater than the sum of its parts.

Photo by Troy Holden, released under a Creative Commons license.

Next Page »