Amazon S3, Elgg and external file handling

November 30, 2006 | 2 comments

For the last couple of days I’ve been playing with two things:

  • An API to allow you to use external file services with your Elgg site
  • Amazon’s Simple Storage Service

Amazon S3 is a simple grid storage service with REST and SOAP interfaces. Essentially it means that you don’t need to worry about hard disk space: you pay 20 cents per Gb, plus transfer fees, and get as much contiguous space as you need.

I’ve got S3 up and running on a test site, and it’s both reliable and quick. Obviously for some tasks you don’t want to be grabbing files from an external server all the time, so Elgg will cache files it handles (for a default of an hour). Nonetheless, it’ll allow you to run vast sites on the Elgg platform at a fraction of the cost.

The S3 plugin just slots into your /mod/ folder. I’ll make it available under the GPL soon.

Meanwhile, if you want, you can now write plugins that easily send files elsewhere. There are several steps to this.

Let’s say we want to carve our files into stone using some stonewriting functions we’ve defined elsewhere. Correspondingly, we’re creating a plugin called /mod/stonewriter/.

Every plugin needs a lib.php in its folder, so create that. The function [plugin-name]_pagesetup() is required in every plugin, so create that – however, we’re not writing anything to the screen, so leave it blank:

function stonewriter_pagesetup() {
}

However, we want to attach ourselves to the file events create (which occurs for a file when it’s been uploaded) and delete (which occurs when it’s about to be deleted). Let’s say we’ll call functions called stonewriter_create() and stonewriter_delete() (for convenience). We can do that with the lines:

listen_for_event("file","create","stonewriter_create");
listen_for_event("file","delete","stonewriter_delete");

(If you wanted, for other purposes, you could also listen for events on objects of type weblog_post and weblog_comment.)

We also want all the files we upload to be written to stone, so we’ll set the default file handler to stonewriter. Additionally, when a file is downloaded by the user in stone tablet form, we’ll use a function called stonewriter_download to return the content.

$CFG->files->default_handler = "stonewriter";
$CFG->files->handler["stonewriter"] = "stonewriter_passthru";

This all goes into the stonewriter_init() function, which is called on page initialisation. So in total we have:

function stonewriter_init() {

global $CFG; // We need access to $CFG so we can set the handler
listen_for_event("file","create","stonewriter_create");
listen_for_event("file","delete","stonewriter_delete");
$CFG->files->default_handler = "stonewriter";
$CFG->files->handler["stonewriter"] = "stonewriter_passthru";

}

Next, let’s write that stonewriter_create function. Functions called using the event handler always take three parameters: $object_type (a string, which will be “file” in this case); $event (another string, eg “create”); $object (a structure that represents the file as Elgg stores it in its database). You always need to return the $object with any modifications. In this case we’ll also delete the uploaded file from the Elgg filesystem, because it doesn’t need to sit there once we’ve written it to stone tablets.

function stonewriter_create($object_type, $event, $object) {
global $CFG;
write_file_to_stone($object);
@unlink($CFG->dataroot . $object->location);
$object->handler = "stonewriter";
return $object;
}

And the corresponding delete function:

function stonewriter_delete($object_type, $event, $object) {
delete_stone_tablet($object);
return $object;
}

Our final task is now to write the function that will be called when we want to get the contents of our stone tablets. The file library will call it with the location of the file in $file->location as a parameter:

function stonewriter_passthru($filename) {
$tablet_contents = get_tablet_with_filename($filename);
return $tablet_contents;
}

And that’s it. I’ve used these hooks to build the S3 plugin using Neurofuzzy’s S3 class, so there isn’t a lot of actual Amazon handling to do.

Even without the file handling, there are clearly lots of things you can do with the event API. One possibility is to alter the contents of a blog post by replacing certain codes with something that’s been defined by your plugin – you might want to write an emoticon plugin, for example. There are plenty of other examples.

This API is in the latest SVN version of Elgg, and we’ll make a new release within the next week. Watch this space.

Audition for Big Brother via a social network

November 29, 2006 | Leave a comment

http://theinternetispeople.com/2006/11/29/audition-for-big-brother-via-a-social-network/

Channel 4’s digital entertainment channel E4 will shortly have its website revamped as a social network. Rather than sift through audition tapes, submissions for the next freakfest series of Big Brother will be via a Youtube-style community video service; presumably they will also find other ways to link their programming output – Russell Brand, Hollyoaks – to the network.

This could be a rather clever move that blurs the line between old-school TV networks and the Internet, while avoiding the highbrow trappings of the likes of Al Gore’s Current TV and C-SPAN’s Viewfinder. On the other hand, they could drop the ball and create something as unoriginal as a Myspace ripoff. We will have to wait and see.

BitTorrent, Bram Cohen and me

http://theinternetispeople.com/2006/11/29/bittorrent-bram-cohen-and-me/

BitTorrent has raised $25 million dollars and lost its founder. Bram Cohen is the guy who developed the protocol, the software and the company, yet they’ve found themselves looking for a new CEO. Comments in the TechCrunch article suggest that it may have been a clash of principles.

What we can’t understand is: presumably Bram has an idea where he wants to go with the software. He didn’t create BitTorrent in a vacuum; there’s a goal and a sort of utopian ideal of what he’d like his software to be like and where he sees it in society. If I was being ousted from a company, I’d probably not tell them everything I have in mind, and very likely this knowledge would come with me to my next gig. It would therefore be in the board’s interests to keep me on.

Unless, of course, Bram’s had enough and just wants to move onto other things. Alas, his Livejournal is mostly about American football, so doesn’t shed much light.

Opera Mini targets social networking users

http://theinternetispeople.com/2006/11/29/opera-mini-targets-social-networking-users/

I’m a several-times-a-day Opera Mini user (it works much better on my Nokia than either the built-in browser or other downloadable options), so I was delighted to see Mashable’s post on version 3. This one emphasises social networking users and more, including a built-in RSS reader. Want to easily keep track of the latest developments in your industry on the move? Now you can.

Opera Mini 3 is available to download here.

Next Page »