I’ve spent the last week in a series of very interesting mind-sharing meetings. First, the American Association of Colleges & Universities flew me into Washington DC to discuss the future of assessment in higher education with education professionals as well as new techsphere friends like Silona Bonewald and Amber Case. Second, Michael Byrne from Harvard University’s Hauser Center for Nonprofit Organizations invited me to speak there about the open, social web, the notes for which I’ll write up and post here soon. A great big thank you to both organizations for inviting me; I’ve felt privileged to have such insightful and interesting conversations over the last few days.

It seems like the shift in innovation in social tools has gone from developing new and interesting technologies to developing interesting models that happen to use technologies. This is a big step, and in some ways represents the space coming of age. There’s still plenty of technological development and innovation to do, but the platform and concepts are at a point where they can be adapted into all manner of social collaborative spaces, business tools, social experiments, games, art projects, and combinations of those things. It’s becoming a very exciting field to work in.

That said, some adaptation needs to happen, and it’s important to realize that these ideas only work in an effective way when they’re made relevant to the outside world. The social web is extremely political: it imposes an opinion about how the world should be open and social, democratic and centered on individual preferences, but ironically doesn’t allow for differences in that point of view. That makes it very hard for late adopters, enterprises, governments and public organizations to feel the benefit.

Over on Persona Prime, Silona makes this point about technology-inspired government transparency:

Where is the change management?  We are doing some big stuff here and we are poised to make serious mistakes and I see no prelim work being done to prevent this.  Where are the best practices in open govt documents?  All I see are “I want” lists.  I have not seen us doing anything serious to ally [the fears of people who might be wary of transparency].

It would be cool if every Fortune 500 company wanted to be on Twitter, but the reality is that they don’t, and often for very legitimate reasons. If what we’re doing is establishing a new, global, decentralized way to create, share, disseminate and discover information, then we have to take into account the differences in all the decentralized nodes. Embracing different corporate cultures, and different opinions on how communication should be, is part of that. Compromising and addressing the fears of companies and late adopters will build a larger userbase for all our tools, and make the platform much more useful in the long run.

It’s exciting to see some of the big names in the Silicon Valley web scene shift gears from evangelizing about the power of the social web to explaining how it can be used to the outside world. For example, Robert Scoble, sometime Microsoft videoblogger and latter day net celeb has started Building 43:

A few people here and there are trying. I watch what Chris Messina, David Recordon, Marc Canter, Joseph Smarr, Kaliya Hamlin, and a group of others are trying to do by pushing a more open web. Those are the kinds of efforts that inspire me and are inspiring Building43. Can we build on what they are trying to do and take it to main street?

Marc Canter is taking it a step further and moving to Cleveland, Ohio, in order to start a new company that helps create Digital Cities:

Where workforce development, content production and local foods meet in the valley of health care, medical digitizing and the history polymers.  Add to that some Seniors interviews, green jobs knowledge bases and authorized venues, community services and common constructs – and you have our project!  Oh yah – and a business directory of……

In both cases, they’re taking the ideas that the web community has created – open, democratic platforms for content agnostic collaboration – and bringing them to communities and people who might not have been exposed to them but could benefit in real, tangible ways. The message I’m getting is that the theory has gained momentum and is rolling into something great; now it’s time to bring it to the world.

And me? I’m in Washington DC this morning, talking with the AAC&U about how these ideas can be used in education.

Blogs.com just published a top 10 list of blogs about the social web that I wrote for them. People are already beginning to comment with blogs they’d love to see added to the list – if you know of one that I missed out, please feel free to add it or link in with your own blog post.

This isn’t, of course, meant to be the top 10 social web blogs ever – these are just some of the blogs that I keep coming back to regularly. (I had more that I dearly wanted to add, but don’t update quite often enough to be considered regular blog resources.) Everyone’s list is different, and equally important in itself; that’s the great thing about the social web.

This how-to isn’t in the usual remit of this blog, but it solves a problem I’ve had for a while – I can’t use the iPhone’s built-in calendar functionality with Google Calendar – so I thought I’d share.

The iPhone 3.0 software update supports CalDAV, an open standard for sharing and updating calendar information. Luckily, so does Google Calendar.

It should really be easier than this; one of the important aspects of integration through open standards isn’t just its possibility, but also its accessibility. This feels more like a hack than real functionality – but at least it works.

1. On your iPhone, press Settings, and then Mail, Contacts, Calendars.
2. Press Add Account… and then Other.
3. Press Add CalDAV account.
4. Follow the instructions for enabling Google Calendar in Apple’s iCal. Specifically, this means using your Google account details for the username and password, and setting the CalDAV server name to be https://www.google.com/calendar/dav/YOUREMAIL@DOMAIN.COM/user.
5. I found that the iPhone didn’t pick up the authentication first time round – you may need to go into Advanced settings and re-enter them. The www in the server name seems to be important.

You can also do it using Google Calendar’s Exchange emulation, but that never worked for me. As with this, your mileage may vary.

Tony Stubblebine has written a great post about the lessons he’s learned from Twitter, which was created at Odeo while he was working there. This advice stands out for me:

Have you ever looked at a piece of social software and thought, or worse, blogged, that it was worthless? Here’s a trick for evaluating social software in a way that isn’t going to make you look stupid six months down the road: assume it’s valuable if people are using it. Then try to figure out what value they’re getting.

I’ve lost count of the number of times I’ve heard people be dismissive about Twitter, or Facebook, or another well-loved web tool because they didn’t understand it. I’ve even been guilty of it myself – but it’s not productive. Much better to figure out why people love it and learn from what you discover.

ScienceBlog reports that on Saturday, Carl Whithaus will announce the preliminary results from a California Department of Education study into increasing academic achievement using computers in 4th grade classrooms (emphasis mine):

During the first year of the two-year study, student achievement increased 27.5 percent, according to Whithaus, who is principal investigator of a study to evaluate the project’s effectiveness.

Computer use – and particularly, online community engagement – increases engagement with formal learning, which is great news for the e-learning software market. But I’m particularly interested in the effect of networks on informal learning – specifically, learning from our activities on the web.

Learning happens when two sets of experiences and assumptions are exposed with each other – in other words, when we communicate. The web is the most globally efficient communications method the world has ever seen, and as a result, I believe, may rapidly transform our world culture for the better.

Last month, I met with J. Nathan Matias from the World University Project, a project that aims to evolve higher education by shedding light on how people learn and teach around the world. His intent is to highlight experiences that people in the west have largely not been exposed to, and in so doing advance mutual understanding between our academic systems. It’s a brilliant idea, which takes advantage of the potential of a universally accessible global communications network.

Recently, the Iranian election swamped Twitter, to the point where they rescheduled maintenance in order to minimize the effect on dissidents in the country. Suddenly, because Iranian dissidents were online and conversing with people from the west, Iran seemed less like a scary, far-off country filled with terrorists and more like – gasp – a country filled with actual human beings. Clay Shirky had this to say:

I’ve been thinking a lot about the Chicago demonstrations of 1968 where they chanted “the whole world is watching.” Really, that wasn’t true then. But this time it’s true … and people throughout the world are not only listening but responding. They’re engaging with individual participants, they’re passing on their messages to their friends, and they’re even providing detailed instructions to enable web proxies allowing Internet access that the authorities can’t immediately censor. That kind of participation is really extraordinary.

On a smaller scale, we’re now interacting with people from other walks of life, with markedly different sets of skills and interests, on a daily basis. The opportunity available to us is not just to get our message out on an unprecedented scale – but to get other peoples’ messages in, and in the process make ourselves more educated and informed than we’ve ever been. On a personal level, it can help us with our fourth grade homework; on a societal level, it’s a revolution.

Following on from my post yesterday on Opera’s new web-server-in-a-browser product, Chris Messina has written a pretty scathing, in-depth critique that also happens to be very smart and on the money. The Financial Times Techblog has an equally skeptical post but misses the point a bit.

In summary: Opera Unite uses the buzzwords of openness, without explaining why they’re useful in a way that makes sense for end users, and without actually being open.

The point Chris makes about users not caring about decentralization without having its follow-on benefits made clear resonates loud and clear with me. Here’s an interview where I talk about data portability and user control – two years ago. Nothing has changed, because nobody’s found a clear way to make this a marketable feature for end users. I’m beginning to think there isn’t one – which isn’t, of course, to say that I don’t strongly believe in the concept. It’ll allow for all kinds of new applications and push the envelope of what’s possible on the web, if we can find the business case for the steps in between.

Meanwhile, I’d love to post a counterpoint. Anyone want to write a post about why Opera Unite is awesome?

Opera just released Opera Unite, a version of their web browser that also contains a built-in web server. As Harry McCracken explains over at Technologizer:

While it’s impossible to judge at this early date whether it’ll “forever change the fundamental fabric of the Web” as Opera promised, it’s a very big idea. Web browsers have always been about bringing information from the Web onto a PC. With Unite, Opera 10 still does that–but it can also fling information from the PC up to the Internet. [..] It launches with some apps that Opera developed itself, including a file-sharing service, a chat room, a music player, a photo-sharing tool, and a note-taker.

Engadget has a video introduction to the application.

This is yet another entrant into the decentralized social web space, but it violates one of my key rules of web application development: keep the browser invisible. Here’s why I think this is important.

I own three computers – two Windows laptops and a Linux machine that runs Ubuntu – as well as an iPhone. All can access the web. At any given moment, I can be connected with any of these devices, depending on which is the most appropriate. For example, I use a 17” laptop at home, but if I’m travelling I’ll take my 12” model; when these are switched off, I might use my iPhone to quickly check something on the web or write a swift email. Additionally, sometimes I connect using other peoples’ machines, or computers in offices I happen to be visiting.

One of the exciting features of the web is that I can use my applications and access my data from any of these. Although I have my preferences as to which device I use, my applications and my data don’t care. They’re agnostic.

As soon as I require a particular browser to be used, I limit myself. I can only access this functionality from the devices that have it installed – which in the case of my iPhone or someone else’s computer is an impossibility. The Opera Labs announcement provides a pretty sound reasoning for decentralized, user-centric services:

Our computers are only dumb terminals connected to other computers (meaning servers) owned by other people — such as large corporations — who we depend upon to host our words, thoughts, and images. We depend on them to do it well and with our best interests at heart. We place our trust in these third parties, and we hope for the best, but as long as our own computers are not first class citizens on the Web, we are merely tenants, and hosting companies are the landlords of the Internet.

However, Opera Unite provides a different kind of centralization and locks us into a particular way of accessing the web. It still yields useful functionality but is a far cry from the cloud-based social architecture that most web application providers are working towards.

Update: To clarify, you don’t need Opera Unite to access services someone else is hosting using Unite. But then aren’t you only half-participating?

When Google Wave was previewed last week, most people focused on its Gmail-like user interface and the slick way it handled collaborative spaces in real time. What was more exciting from a web technology point of view, although much less sexy, was the underlying protocol: Google Wave is built on top of an extended version of an open source real-time data standard called XMPP (aka Jabber when it’s used as an instant messaging context). This means that anyone can build servers and end-user applications that are compatible with Wave.

Yesterday, the BBC announced that they were using an open source framework called Hemlock in order to provide real-time web games on their Childrens’ site. Hemlock’s interfaces are built in Flash, but the real-time aspect is powered by XMPP. Developer Ron DeVera explained why over on the Hemlock blog:

XMPP is commonly used to enable chatrooms, like Gtalk or Facebook Chat. However, instead of just sending text, Hemlock lets you send all kinds of data at the speed of instant messaging. Even better, XMPP is designed to scale up and send data to many people at once, so you can push that data to multiple people in real-time.

In other words, it’s perfect for real-time applications on the web.

Traditionally, live web applications have used a development methodology called AJAX, which stands for Asynchronous Javascript And XML. In order to provide the illusion of a real-time application, the web browser needs to continue to ask the server if there’s any new data to ask, via a new AJAX request – which is a waste of bandwidth and resources both on the user and server side. XMPP, on the other hand, pings the client software whenever there’s new data.

Servers already need software to host and process dynamic web pages, as well as a database server to handle most types of user data. To use XMPP, you need to add an XMPP server to the mix – something that in the past was more fiddly than most people were willing to play with. However, times have changed, and ejabberd is free, open source and easy to install on Windows, Mac OS X and Linux servers. (I’m still at the early stages of playing with it; I’ll let you know how I get on.)

The XMPP community is still very small compared to the wider web, but developments like Wave and Hemlock should accelerate growth and interest in it. There are some interesting projects springing up, and lots of libraries available for XMPP development; XMPP programming is a skill developers should get in on now, before it really heats up.

I believe a truly decentralized social web is required to fulfill the web’s potential as a platform for business collaboration, and I’m very interested in helping to push the technical and conceptual boundaries in that direction. I spend a lot of time on this blog writing about that, but I think it’s also important to remember that a huge amount is possible using the technologies, standards and ideas that we can currently pick up and use.

Creating a new web tool, or adapting one for your own use, can be a bit like pitching a movie: a lot of people come to me and say things like, “it’s like Delicious meets Youtube, but for the iPhone”. That’s great, and can result in some very interesting ideas, but I think it’s always best to go back to first principles and ask why you need the tool to begin with. My post The Internet is People addressed some key points on this:

• Your tool must plug into an existing network of users, or be useful for user 1 (the first user to sign up). Delicious lets you save your bookmarks into the cloud; Flickr lets you easily upload photos so other people can see them. Both services come into their own when you connect with other users, but the core of the site is useful before you’ve done so. Facebook is different, but it had the Harvard real-world social network to plug in – and it now acts as a useful aggregation of your other activity on the web, which arguably is useful for user 1.
• You can’t build a site and assume people will come and use it. It’s a lot of hard work, even when the technology is ready for launch; you need to lead by example, constantly adding content and using the site as you would like it to be used. Not to mention the hours you have to put in promoting it elsewhere.

The feature set itself should be tightly focused:

As each tool should focus on one particular network, or at least type of network, I’d argue that the exact feature set should be dictated by the needs of that network. Educational social networks might need some coursework delivery tools; a network for bakers might need a way to share bread recipes. The one common feature in any social network is people; even profiles may not be entirely necessary.

I mention at the end of the post that these principles were the guiding ideas behind the design of the Elgg architecture. They’re now the principles behind the tools and strategy I develop for my clients.

In this blog you’ll find lots of talk about new technologies, innovative approaches and the ethics of social media. These allow us to build interesting new tools, but they always sit on a firm foundation: the Internet is just people connecting and sharing with each other, and the purpose of web tools is to make that as easy as possible.