Making billions of dollars from the federated social web

August 28, 2012 | 4 comments

Diaspora was pretty exciting. A Kickstarter campaign that promised it all: a platform created “for everyone to have full control over their data and to empower people in to become responsible, secure, and social Internet dwellers”. They raised over $200,000, which at the time was the largest Kickstarter funding round ever. Neat!

Yesterday, Diaspora announced that they were becoming a community project. That means that the wider open source community will be responsible for its direction; the original team will be involved to some extent, but are now involved in projects like Makr.io, which just made its way through Y Combinator. (Makr.io is released under the same open source license as Diaspora: here’s the GitHub repository.)

The federated social web is a holy grail for many of us. It’s a simple idea: rather than ceding control of our data and activities to a monolithic, monopolistic third party, why don’t we create an open market for services we can use, which can all interoperate? Those of us willing to set up our own servers would own our own data; others would use the servers set up by their schools or employers; others still would join the hosted service of their choice. The result would be a more innovative, web-like application web, where niche interests could be better served and people could choose the applications and interfaces that best suited them.

It goes far beyond Facebook-style social sites: the potential is for deep interoperability between every application on the web and beyond – and, potentially, identity spaces where we could keep our personal information safe.

Back in 2005, I tried to make the case in a letter to the social web hating journalist Andrew Orlowski:

If a thousand sites depend on Flickr, what happens if Flickr goes down? Wouldn’t it make a great deal more sense to think about standards for data transfer and availability to allow for Flickr-like data all over the Internet, rather than in one place – effectively a peer-to-peer network (or networks) of tagged resources? That way one can still grab information and manipulate it, but with zero dependence and a high tolerance for network failure. Rather than there being one nebulous Microsoft Office 12 For Web, there could be ten thousand office applications that all use the same standards and allow for data transfer perhaps with storage services elsewhere. The answer to “how bad do you think it will get” is “exactly the same as now” if we allow one service provider to create a monopoly for a particular type of service. As ever, the solution is open standards with a framework that anyone can use. This is obviously not going to lead to billions of dollars for any one provider, so it’s probably up to the open source movement to create.

Returning to that letter seven years later, I think I was wrong about one thing: I believe it is possible to make billions of dollars from interoperable social web applications – and there’s no need for mass public adoption of apps using federated social web standards to do it.

I’m convinced that business software should be using federation protocols, as long as those protocols have built-in access permissions. Imagine if your organization – a government department, say, or a corporation – could selectively create shared spaces for joint projects with other organizations, for as long as the project existed. Imagine if you could collaborate in more fluid ways, where applications were more like documents that you could introduce into a project when you needed them. Imagine if email was smarter. Whereas decentralization in public consumer apps is an ideological feature, in enterprise apps it has a solid business case and adds measurable value. Businesses will become more profitable by using federated apps.

On hearing the news about Diaspora, StatusNet’s Evan Prodromou made a very sensible plea:

I only have one favour left to ask: please, for the rest of us who are still working full-steam on federated social networks, don’t fall into the comforting fiction that the problem is insurmountable. It’s not; it deserves our attention and support. You didn’t waste the last two years on something pointless and unattainable. Your work matters.

It might sound like hyperbole, but I still believe it could change the way we do business, and ultimately how we all communicate.

Update: In the comments, Johannes Ernst points to his post about why decentralized software is 10x harder. I think he’s right about the problems that need to be overcome. But check out his note at the bottom: watch this space.

Off the hooks: user-centered development in a mobile-first world

June 11, 2012 | 1 comment

The following post contains my notes for a talk I gave at Over the Air 2012 at Bletchley Park on June 2nd, 2012. They build on my notes for a previous talk, Building the user-centered web.

Desktop isn’t enough.

For the last few years, I’ve been working on latakoo, a service that lets you share video fast using any Internet connection. Video is, traditionally, a pain to upload, due to very large file sizes, combined with the limited upload bandwidth inherent to most commercial connections. We solve this by compressing the video first, using a very simple interface: just drag, drop, click, and your video is uploading, and shared in a private, enterprise environment. There’s no need for any substantial training or technical video knowledge. It just works.

We support a lot of different video formats, and the resulting file is often 1-4% of its original size, while remaining very high quality. A lot of TV stations use latakoo – the Science Channel used us to send footage of the SpaceX launch back to base – but we also have integrations with Facebook, YouTube, Box with more to come. latakoo’s a useful service, and a great team that I’m proud to be part of.

However, an app that runs on a desktop or laptop computer isn’t enough. Sure, the latakoo app works great in those environments. But a lot of people began asking us if we could make it work on mobile – including those TV stations. TV news stations are arming their reporters with iPhones and sending them out in the field instead of laptops.

Video compression is no problem for my i7 MacBook Pro – but can it work on my handset?

Of course it can.

latakoo iPhone appMobile devices are this century’s personal computer.

I have latakoo apps on my Samsung Galaxy S2 (in beta, coming soon), my iPod and my iPad. All of them are very comfortable compressing video – because all of them are relatively powerful mini computers.

My Samsung Galaxy S2 has a declared clock speed of 1.2GHz and has 1GB of RAM; the Galaxy S3 has four cores at 1.4GHz, and 2GB of RAM. It wasn’t so long ago that laptops were being sold with those specs, and it’s no accident that mobile growth is outstripping the rest of the computer industry:

Oh, and here’s one more quote:

“We do not currently directly generate any meaningful revenue from the use of [our] mobile products, and our ability to do so successfully is unproven.”

That’s from Facebook’s IPO filing. Over 50% of Facebook users access the service through a mobile app or site – yet, until the last couple of weeks, they had no way of obtaining revenue from these accesses. (Their promoted posts feature is transparently a way of obtaining revenue in a way that applies to mobile users.)

As we know from Building the user-centered web, the web has transformed from a series of context-free, stateless pages to interlinked applications that have social functionality at their core.

Curiously, despite this social focus, each application is its own atomic destination with its own URL, and is by default only aware of data created within it. That means we need to register for each application we want to use, fragmenting our accounts over potentially hundreds of products and company data centers, and that the documents, files and data we create within them can’t easily be shared with other applications. Currently, each web application is like a silo: they exist on their own, and if they interoperate at all, it’s through specific links between applications that have to be individually developed.

Mobile-first social apps exist, and are beautiful – but are being designed to this silo model. Path, Instagram, Viddy and Socialcam are four examples of gorgeous apps that are designed to trap usage in their own databases, and often spam users into joining. After all, the number of users in your database, and growth thereof, is the predominant measure of value in Silicon Valley. Instagram certainly wasn’t worth $1 billion to Facebook for their revenue.

SilosSilos stifle collaboration.

Collaborative spaces in social web applications are like documents: they’re one of the currencies of the social web. Just as I need to be able to use my word processor of choice to edit a document, I need to be able to use my social tool of choice to collaborate with others – and if I want to share a collaborative document with someone, I should be able to do so regardless of the social service they happen to use. Where I choose to host my identity should not dictate the kinds of content I can create. Not to mention that some companies simply can’t use existing social platforms, for ethical and/or legal reasons. The UK’s Data Protection Act, for example, precludes the use of Facebook in schools.

Right now, we have to register with each application we want to use. What if we required each application we used to register with us, in digital identities under our own control? What if, using these identities, anyone could connect to anyone else, and anyone could store their data anywhere as long as the storage provider followed the same broad standards?

The web would become truly social.

Your ability to collaborate is not subject to a single company’s success: social functionality and application infrastructure are inherent in the network itself. The possibilities for collaboration are not subject to technology beyond common open standards, which can evolve. A wider range of application possibilities is ensured, because web applications gain the ability to interoperate in a general way. Finally, privacy and user control are established by allowing a person to determine which application has access to which data.

Furthermore, when you remove the silo, private or short-term collaborative spaces become possible:

  • Transience: there’s no need for the community to exist for longer than it has to.
  • Simplicity: there’s no effort involved. Once you’re done with a community, you simply close the communication.
  • Privacy: it’s very hard to share activity with the wrong people.
  • Decentralization: the community is physically hosted between all the involved parties, removing single points of failure.

This user-centered future is being built.

One of the next big hopes is Web Intents. Here, the user performs a generic action in an application, like “share”, perhaps by pressing a button. The user can then choose from all the services he or she prefers, that support this particular action in this context. The list is built automatically from the user’s preferences – a set of applications which are, indeed, registering with the user’s central identity.

Colour coordinated appsThis user-centered future has been built – on mobile.

Web intents are, of course, an adaptation of Android’s intents model. (Albeit in a simplified form.) And, in fact, we already do all kinds of identity-like things with our mobile devices.

  • We can pay with mobile.
  • We can authenticate with mobile (as many of us do with online banking).
  • We organize our contacts, our calendars, our messages, our photos and our videos on mobile. We speak to our loved ones on mobile. We read and watch and play on mobile. We write the identifying address of our mobile devices on our business cards, and have done for years.
  • Mobile knows your context: where you are, and who you’re with, where you’ve been, and where you plan to go. It’s far smarter, and far more social.
  • Mobile is already a decentralized social application platform, with a wide range of communications methods, from SMS through Bluetooth and NFC. We can use all of these methods, depending on the context, to send social information.

Smartphones already let us share in a decentralized way, but only specific, agreed data types like SMS, MMS and email. So, how can we expand on this to send arbitrary social data?

Sending social data on mobile devices

It turns out that the Activity Streams standard allows us to handle social data in a generic way, that can carry specialized data for niche application types. For example, a task in a distributed enterprise system might be an extention of note, one of the base ActivityStreams object types.

ActivityStreams is an extension of Atom, so it could contain the content itself – although this might not be feasible on mobile. We could send mobile notifications with ActivityStreams-like payloads (up to around 1400 characters on iOS, or around 1024 bytes on Android). If an application handles a particular object type, great! Otherwise, perhaps this is a great opportunity for the platform provider to gently point to their app store to make some suggestions.

This could be a solution for direct notifications, but still need to read updates from a central list.

Abstracting from individual mobile devices

I carry two phones: one for the US, one for the UK. I also carry an iPod for when I have to use an iOS app, which has its own set of profile data, although fewer communications options. And indeed, because of contracts and other factors, people upgrade their phones every 18 months to two years, possibly changing their numbers. Handsets get stolen, and some people have entire bags of SIM cards that they use on their travels as appropriate. And finally, backups are important.

An abstraction service – like a social version of Google Voice – may help here, gathering messages in a feed and sending notifications to handsets, or allowing handsets to retrieve as a batch.

Once you’ve done this, the obvious question is: could this model work on the desktop or in a pure-web model? Sure, but by thinking mobile-first, we get to build on work that’s already been done on these platforms. Mobile platforms lose a lot of the legacy baggage inherent in both desktop operating systems and the web, and they’ve been designed from the ground up to support distributed communications in an intelligent way. This doesn’t diminish the importance of the web as a cross-platform home for both applications and data, but it provides new opportunities for linking people data and building next-generation social applications.

Everybody benefits.

Platform owners get to build more engagement – and therefore value – into the platform itself.

Application owners get new ways to spread their software – through organic social proof.

Users get freedom from the silos that stifle their possibilities for collaboration and creativity.

Some questions:

  • A lot of people are still on Symbian and other legacy mobile operating systems (notably all those Indian users). How can we reach them?
  • How could we prevent spam and ensure data privacy on such a platform?
  • Which would be the best platform to start on? I think Android technically has the most support for this model, but Windows Phone may have the most organizational support.
  • In an era where platform value is so closely tied to trapped engagement, would anyone fund this?

Occupy Facebook: innovation in the era of social protest

December 29, 2011 | Leave a comment

Occupy UMWired reports that a group involved in the Occupy movement is working on its own decentralized social networking platform:

“I don’t want to say we’re making our own Facebook. But, we’re making our own Facebook,” said Ed Knutson, a web and mobile app developer who joined a team of activist-geeks redesigning social networking for the era of global protest.

Dave Winer notes that it probably won’t succeed:

There is no market for that. Facebook is the Facebook for the 99 percent. The goal should be to make something open and non-monolithic that provides many of the most valuable services of Facebook without the silo walls. It should not be something that an individual does, or a small group laboring heroically, rather it should be something that the Internet does.

In principle, I agree with Dave. It’s a fundamental mistake that a lot of people make when they build a social site. I used to spend a fair amount of my time at Elgg dissuading people from trying to make the new MySpace (which was the hot thing at the time), and inevitably, none of the neo-MySpace generic social networking sites worked out. Where open social networking did work out was for specific use cases: connecting people inside charities, providing platforms for reflective learning in education, rallying around causes or products.

However, that’s also an argument for why a social engine for Occupy might succeed. If it’s geared specifically at the needs for protest, while using existing technologies and simple solutions where appropriate standards don’t exist, it may become the first broadly-adopted decentralized social platform. From the backbone of Occupy using it, I can easily imagine other progressive organizations and individuals picking it up, eventually spreading through the academic sector – until it finally reaches the commercial world. Sure, there’s no market for that, but there’s a drive and motivation.

In an age where laws like SOPA are considered – where the threat of sites being yanked off the Internet without due process is real – decentralization in platforms underpinning progressive causes makes a lot of sense. You can easily imagine the “retweet” or “reshare” feature found in the likes of Twitter or Tumblr being used to create a copy of information so that it spreads across the network like an old-school Usenet post; a kind of social multicast which ensures that nothing really dies. Some nodes may live in the cloud, some on dedicated server clusters, and others on always-on home computers linked via a Comcast router; the same free speech, everywhere, indelible and impossible to control.

That’s a future I can get behind, and a platform I’d love to be involved with. In comparison, taking on Facebook just seems so small.

Occupy photo by Sasha Y. Kimel, released under a Creative Commons license.

Identity, contact management and federated social networks

October 24, 2011 | Leave a comment

Last week, Doc Searls reflected that everything being worked on at the Internet Identity Workshop is meaningful to CRM:

It just occurred to me that everything being worked on at IIW is meaningful to CRM. I had been thinking that only the VRM stuff was meaningful, but I realize now that all the IIW stuff is, because — from a CRM perspective — it’s all about customer empowerment. And empowered customers are entities that CRM will welcome, sooner or later.

I think that’s true, but I also think we can go further than that. A huge part of identity on the web is controlling who can see what: think about the Google+ Project’s approach, where your identity consists of a series of data objects (posts, photos, status updates, etc), each having its own set of access controls. Controlling access to items requires that you have people to restrict access with. Therefore, contact and relationship management is integral to digital identity.

In turn, federated social networks are integral to both. For identity to be useful online, you need to be able to use it virtually anywhere. If identity is a series of items restricted to people based on your relationships with them, those relationships need to persist everywhere you use your identity. Hence, your relationships need to federate across identity-aware applications – and federated social software is the future of identity online.

Next Page »