Tracking twitter for dev8d

In the run up to Dev8D, there’s been a lot of twitter activity, which will hopefully continue throughout and after the event. I’m hoping that something useful can come out of it, so I’ve started a running a couple of scripts based around this.

Capturing twitter posts

Firstly, I’m simply capturing all tweets which mention ‘dev8d’ using twitter’s search API.

I’ve written a small python script to do this, available here:
archive-twitter-search

It’s designed to be a quick solution to archiving twitter posts related to events (or anything with a keyword), with zero configuration/setup (you do still need to specify a search term though…).

Every 10 minutes, any new tweets mentioning dev8d are stored in a SQLite database. They should be easy enough to query or export to another format in this way.

Capturing the twitter network

The second (and potentially more interesting) thing I’m logging is the change in the network of dev8d attendees on twitter.

I’m capturing all users with a twitter account who’ve registered on the Dev8D wiki, and any user who posted/received a tweet mentioning dev8d.

I’m then periodically (as often as twitter’s rate limit allows) getting the list of friends and followers for each user, and logging when any changes to these lists occur.

By joining users and lists of friends/followers together, it should be possible to build up a picture of the connections between developers attending dev8d, before, during and after the event.

I’m hoping that over time (well, over dev8d at least), I’ll be able to see how the network of attendees changes, expands and grows together.

The data will all be freely available (upon request for now – I’m running the scripts on a desktop PC), but I’m planning to add the data to data.dev8d.org after the event.