The serendipity of knowledge and how we can capture it

One of the things I really like about America is C-SPAN.

If you’ve never sat around and watched TV here, you might not be aware of it: the cable TV operators in the United States got together and created public service channels that operate a bit like BBC Parliament in the UK: they show speeches by people in government, but they also show talks by authors promoting their books, and more general talks by academics and researchers. This weekend I saw a talk about the future of print on demand in bookstores and one about the propaganda surrounding the war in Iraq.

Right now I’m sat with my laptop in front of a talk at the Heritage Foundation about the differences between secular and religious communities in terms of how much money they give to charity. As it turns out, as a percentage, people in South Dakota give a huge amount more to charity than people in San Francisco – something I wouldn’t have thought about. Religious people are twice as likely to give blood. This is a very important aspect to our society, and something that goes against my personal grain. (I’m affronted, but then again, when I earn a salary I make sure I give 10% of it away to charity: a very religious concept.)

This is definitely a talk I wouldn’t have gone out of my way to see, but it’s very interesting, and I’m glad I’m sitting here and watching it.

It leads to an important question in the context of online communities: how can we build in the same kind of serendipity into systems where, traditionally, you search out specific pieces of information? Some of the most interesting things I’ve learned, I’ve learned through television or through people talking about stuff they happen to know about. If we’re filtering things down according to our interests, we may never find out about that same stuff: things that could enhance or enrich our lives, but aren’t what we’d go out and specifically look for.

Wikipedia has a very simple solution to this: the random button. It’s possible – and I’ve done this – to sit down and just read random Wikipedia articles all day. You learn about all kinds of different things, but what if you want to narrow the focus slightly? If you were to have a random button on an Elgg system, for example, you would get cartoons, pieces of personal blogging, links to educational texts and diagrams from education. This might be interesting for a while, but what if you wanted the C-SPAN (or NPR) experience? What if you just wanted presentations or documents about interesting things?

Google has half the solution to this: as most of us know by now, documents largely rise in search results due to the number of other sites that have linked to them. This assumes that if someone finds a resource useful, they’ll want to bookmark it and share it with other people; it’s a reputation filter. Unfortunately, there is a flaw in this approach, which was initially exploited by the Church of Scientology and is now routinely used by common or garden spammers: not all distinct sites are unrelated. Therefore, any organisation can create a bunch of different sites that run on different servers, tightly interlink them, and watch their sites rise dramatically. For similar reasons, social networking sites artificially rise to the top of the rankings, which is why, if you have an Elgg profile, it’s probably the #1 search result for your name.

We’re developing a ‘favourites’ system for a client, which will eventually find its way into the world as a GPL release for Elgg. Using an adaptation of the Google model, it’s possible to determine which resources in a social networking system are best on a particular topic depending on how many times they’ve been favourited. This is particularly useful if you want to look for, say, a really good presentation on pedagogy; but you can also use it to extract other information, like the resources on all topics that have been favourited the most – both in a system as a whole, and by the people you’ve connected to. Suddenly serendipitous discovery comes rushing back into play, and may become the most interesting part of an Elgg site. It’s something that I’d welcome in all social networking sites, particularly bookmarking tools like

In the meantime, if you’re interested in C-SPAN, they’ve very cleverly made lots of their content available to stream online – something I wish the BBC would do in the UK. I love television, and I think, as Internet community builders, we have a lot to learn from it. If we can marry its menu of interesting content with the democratic elements of web 2.0, we will have created a very interesting kind of resource.

Perhaps because they’re dealing with a clearer television metaphor, the sites that handle this the best are Youtube, Google Video, and Odeo. Splashed across their front pages are the most popular content, both streamed into channels and generally – it’s very easy to stumble across content you might not ordinarily have watched. In the case of Youtube, these are videos generally popular to the 14 year old set, so they might also be videos you may never wish to watch again. Nonetheless, for their audience, it works. I think this would be a very interesting concept to harness across all types of content.

Leave a Reply

Your email address will not be published. Required fields are marked *