Learning in hallways (with APIs)

November 18, 2012 | 4 comments

I can’t let Clay Shirky’s piece, Napster, Udacity and the Academy, go un-commented-on:

Open systems are open. For people used to dealing with institutions that go out of their way to hide their flaws, this makes these systems look terrible at first. But anyone who has watched a piece of open source software improve, or remembers the Britannica people throwing tantrums about Wikipedia, has seen how blistering public criticism makes open systems better. And once you imagine educating a thousand people in a single class, it becomes clear that open courses, even in their nascent state, will be able to raise quality and improve certification faster than traditional institutions can lower cost or increase enrollment.

You can – and should – read the whole piece here.

I completely agree with it, and I think that startups like Udacity will broadly be a good thing for the world. (Of course, it’s worth mentioning that this is a movement that OpenCourseWare started a long time ago.) Having said this, there are a few important tenets about learning that I think aren’t necessarily captured by the Udacity model.

  • There are different kinds of learners.
  • Learning with your peers is important to some people – and learning alone is important to others.
  • A one-size-fits-all approach doesn’t work when it comes to education.

I went to public (state-funded) schools, and a public university. I can’t claim to have experienced the one-on-one education that you might receive at Harvard, for example. But it’s nonetheless true that education has traditionally been, at least a little bit, tailored; you could always go to your Director of Studies if you had a problem, or talk to your professor about substituting work or taking a different focus.

In the modern web age, by which I mean from about the iPhone onwards, we’re used to cookie-cuttering users. Everyone gets the same interface, in the same design, with the same content types, because that design is good, it’s efficient, and don’t you love good design anyway? We’re all supposed to write a certain way, consume a certain way, look a certain way.

Applying this principle to education will be disastrous.

There’s a lot wrong with education right now, particularly in countries like the United States and Britain, where class systems are enforced through high fees and barriers to entry. But in a knowledge economy, we should be emphasizing creativity and individual strengths, rather than attempting to make learners fit an ever more rigid, dehumanizing template. (We should be doing that with users of our applications too, of course.)

But as I said at the beginning, I don’t think this is a bad trend. It’s also an inevitable one. Educational content will be open, it will be delivered en masse, and you will be able to access it from anywhere in the world. It will be a great thing.

The trick is how you consume it.

You can use Udacity’s interface, if you like. But just as I have the freedom to take three classmates to the pub (I went to university in Britain, remember?) and talk over our notes there, I should have the freedom to take some of my classmates and discuss on Facebook, or a collaborative Google Drive space, or on some other custom platform.

And that’s where the technology focus becomes really interesting. Web applications have APIs: Application Programming Interfaces that let other applications talk to them programmatically. The same API approaches that allow people to build third-party Twitter apps or to sync Instagram with Facebook could allow people to take streams of learning from the learning service – let’s say Udacity – and pull them into the platforms of their choice. Other commercial applications, or freely-available open source projects, could take that learning and allow you to interact with it individually or in a group. And then you can use the app or method of your choice to submit your work back to be evaluated. And if everyone’s using the same APIs, then everyone benefits: learners get to pick and choose their courses, and the educational providers get to participate in an open marketplace that’s as big as the web.

In this model, the raw course is always the same. But suddenly there are a hundred thousand lenses that you can apply to it, so if you’re a visual learner, or a group learner, or a solitary text-based scholar, you can find the interaction method that appeals to you, pull in relevant third-party information and conversation to augment your learning, perhaps even talk to third-party tutors in other countries (or next door), and have a much deeper, richer, more personalized experience than you could ever have had before.

My worry with the new educational startups is that they’ll try and lock themselves down, in the way that Twitter and Facebook have locked themselves down. If, on the other hand, they can open up and embrace what the web really is, there’s the potential for a real revolution.

The Edinburgh Festivals API: a case study in innovation

May 14, 2012 | Leave a comment

I was the inaugural Geek in Residence at the Edinburgh Festivals Innovation Lab, working with Rohan Gunatillake and the Edinburgh Festivals on open data and digital accessibility.

Here’s a video from Rudman Consulting for AmbiTIon Scotland about the Festivals API portion of the project:

httpID: adding identity to standard HTTP requests

April 19, 2011 | 17 comments

This is a more technical post than I’ve been writing lately. I’m considering splitting out into two blog channels; let me know if you’d prefer this.

This is a request for comments and ideas. Please let me know what you think in the comments. Thanks!

One of the advantages of the decentralized social web, as opposed to a social network (federated or otherwise), is that identity can, theoretically, be shared with any web page, anywhere. That page doesn’t have to be running any particular software or provide any particular function; it should optionally be able to support identity-related features. That could then be used to tailor the page to the viewing user. (Of course, sharing identity should never be required, for security reasons.) This is part of three broad activities that I see as being part of the social web:

  • Publishing web content in an identity-aware way
  • Consuming web content in an identity-aware way
  • Sharing socially

Much of the decentralized social web development activity to date has been focused on the third point, and on reading and writing as part of a social web application like StatusNet or Diaspora. However, I’d like to look at the first two points with a view to make them web infrastructure, rather than features of a web application.

To achieve this, I’d like to be able to report, as an option, the identity of the person making an HTTP request, as part of the headers to that request. This might come from the browser itself, eg via an identity plugin, or it might come from a web-based identity proxy.

HTTP supports basic authentication, which involves sending a username and password, potentially in the clear. Out of necessity, we’ve moved beyond this, eg for things like API authentication. Often tokens, hashes and encrypted requests are included as extra header values to authenticate a request.

I’d like to use the same general principle for identifying a user. Here’s how it might work:

  1. The user visits a site for the first time. The browser sends a standard HTTP request. (Or, alternately, a HEAD request, if the site content isn’t required.)
  2. The site responds as normal, but with an extra HTTP header indicating that it’s identity-aware, including the URL of a handshaking endpoint. This will be ignored by clients that aren’t looking for it.
  3. If this is a standard browsing scenario, the user’s browser asks if he or she would like to share identity information with the site. For the purposes of this example, the user clicks “yes”. (This step can be left out if this isn’t a standard browsing scenario.)
  4. Via the handshaking endpoint from step 2, the user’s browser gives the site a public and private key, and a URL, through which it can access the user’s identity information as an XRD file (as in Webfinger). This is exactly the same as the public and private key system used to retrieve social information in points 5 and 6, using the same method. The site simply makes a signed request to the user’s identity URL, which can be anywhere.
  5. The browser receives public & private keys for use with this server only. These might be stored in the browser, or in some central identity store that all the user’s browsers access.
  6. Whenever the browser makes a request to the server, it adds extra headers using these keys (and HMAC-SHA-1), signing each request with the user’s identity until he or she says otherwise. It also sends a header to indicate when the user’s identity information was last changed, in order to prompt the site into obtaining new information if it needs to.
  7. If the site in point 4 is associated with a specific person (for example benwerd.com would be associated with Ben Werdmuller), he or she can use the public and private key generated in step 4 to browse the user’s site.

The publisher would get a list of users who have identified with the site, and, depending on their server or content management system, might add some of them to special access control groups that would allow access to different content. The next time the user visited the site, they’d see more privileged content. A notification would probably be sent to them to let them know this had happened, but this is out of scope for what I’m discussing here. (Perhaps notification methods could be shared as part of a user’s identity information?)

Conversely, the user’s XRD file containing their identity information can also change depending on who’s accessing it (as the requesting site always makes a signed request).

This system has a number of advantages:

  • It’s server and system agnostic. It simply uses the building blocks of the web.
  • It’s very easy to build for. Checking and setting HTTP headers are easy to do, and don’t require any front-end work like HTML parsing or JavaScript libraries. This makes it usable for APIs and feeds as well as web pages, and for clients that use web APIs as well as web browsers.
  • The web isn’t just a platform for people to read these days. This method doesn’t depend on anything visual.
  • You don’t need to control the root of a domain to make it work. If you install a script at http://yourdomain/~foobar/banana/hockeystick.php, the system will be happy there too.
  • It’s passive. There are no blockers if you don’t supply identity information – you just see something different.
  • It’s based on similar assumptions to WebID, but doesn’t require SSL certificates in the browser, and it’s as easy for a web app to implement as it is for browser software.

It incorporates the following assumptions:

  • Relationships are assymetrical. (Here, there’s a set of keys for each side of a relationship. If one side stops participating, perhaps by removing the other from an access control group, the other side is still valid.)
  • Privacy isn’t binary. (Everyone gets a different view on a given page or piece of data.)

Let’s call it httpID. I’m looking for feedback on the idea and process. Does it make sense? Have I missed something obvious? Let me know. If there are no major blockers, I’ll firm up the spec and create some libraries.

The Open Stack and truly open APIs

May 28, 2009 | Leave a comment

The following is an expanded version of the short talk I gave in Oxford yesterday. My original slides follow the transcript.

There’s a new kind of web development afoot, which marries old-school object-orientated programming techniques with the distributed power of the web.

Application Programming Interfaces (APIs) are published sets of instructions for programmatically querying or extending a service. For example, the Google Maps API allows anyone with some development skills to build applications based around Google Maps; the Twitter API allows anyone to build an application that lets you interact with Twitter (like Tweetie or Tweetdeck). The act of allowing anyone to do this is generally thought of as being “open” – as in, open to anybody. While that’s true, in another sense they’re very closed.

The trouble is, if you write a microblogging application using the Twitter API, you’re locked into Twitter. If you want to write an application for another microblogging service, you have to use their API and start from scratch again. The formats produced by each service’s API are proprietary, as are the methods to query them. They’re incompatible with each other by design, because those services don’t really want you moving users’ data around between them. They want users to sign up with them and then stay there – a great proposition for the service, but a lousy one for the users, who have no real way of importing or exporting the content they’ve created.

Furthermore, there are some situations where this one-API-per-service model breaks down completely. Let’s say you’ve joined a social network and you want to check to see which of your friends is already there. The service provider could use a Gmail API to check your address book, but some users will use Hotmail for their email, so they’ll need to use the Hotmail API as well. Repeat for Yahoo, and every single email provider under the sun – there’s no way anyone could possibly write this feature without a generic API that works across all services.

Enter the Open Stack, which is a set of generic APIs designed to provide common tasks:

  • eXtensible Resource Descriptor (XRD): allows an application to discover resources and APIs relating to a particular page (for example, an application could check a user’s profile page and discover that they have an OpenID). There’s no point in having open APIs if an application can’t find them, and XRD fills this gap.
  • OpenID: allows anyone to log into any service that supports it using the same digital identity (potentially linking that user’s accounts on those services together). A WordPress.com ID is an OpenID, for example; you can log into any OpenID-compatible site with it.
  • OAuth: a way to authenticate access to API-based services without forcing the user to present their username and password for that service. Previously usernames and passwords were passed in order to authenticate APIs, which led to serious security issues. Here a user can easily and securely grant or deny an application’s access to their data. OAuth can also be used to apply granular access controls to a piece of data.
  • Activity Streams: a way to broadcast your activity on each service (for example ‘Ben has published a blog post’, a la Facebook’s river) so that it can be aggregated at a central point.
  • Portable Contacts: a simple way to transmit collections of contacts, like email address books and lists of friends, between services.
  • OpenSocial: provides a framework for hosting remote application widgets on a service.

In programming jargon, a stack is a specific data or platform structure; actually, this is more of a pile of useful technologies that can (in part) be used together.

These generic APIs allow for a more distributed kind of web application. Suddenly, instead of writing support for a particular function from scratch, you can pick up an existing code library and slot it in. You can interact with any service that supports them without writing any further code.

However, there are some unresolved issues that need to be discussed. One major headache is the privacy implications. If I sign up to BensFabApp.com, I agree to the terms of service and privacy policy there. However, behind the scenes it might be accessing the APIs for AppOfEvil.com, which has a very different privacy policy and terms and conditions. How does that second set of terms and conditions apply, and what’s the legality of passing a user’s data across two services with two different sets of terms, when the user only knows about one of them?

A second issue is a user’s control over their data. When BensFabApp.com sends data to AppOfEvil.com, it may get cached and duplicated. What happens if I delete the original data on BensFabApp.com? So far in the open stack there is nothing to handle deletion of content. It’s a fair argument that when something is published publicly you lose control over its distribution; however, if access has been limited using OAuth or another method, there’s no way of reaching out an ensuring its removal from every system it’s been sent to. That would be fine if the data had been sent to another person or application with the user’s express permission, because they’ll be aware of the implications. However, if it’s been sent completely behind the scenes, they have no way of knowing that it was sent to begin with.

These are issues with any API-based service, but generalized APIs are likely to be used more frequently. Solutions to all of these problems will be found, but it’s important to note that they’re not there yet – which serves as a warning to applications developers and an opportunity for anyone who wants to step up and provide them. As this kind of open API becomes commonplace, new kinds of web applications will begin to emerge as developers spend less time reinventing the wheel and more time innovating. It’s just one of the things that makes the web as a platform so exciting to work with.