WARNING: long blog post!

I’ve been promising a number of people a blog post on how I archive tweets. I set archives up for lots of reasons – often for an event I am attending to record tweets to refer to at a later date, or sometimes for projects I am involved in to keep a record of conversations. There are a number of different methods of archiving tweets, some of which are outlined below.

NB: This post covers archiving tweets made using a specific hashtag from all users, not archiving personal tweets.

  1. TAGS (Twitter Archiving Google Spreadsheet)
  2. Twubs
  3. Eventifier
  4. Tweet Archivist
  5. TweetDoc

There is also HootSuite (with Pro subscription), but I’m just focusing on the free options in this post.

My preferred tool at the moment is TAGS, but in order to try some other options out, I set up archives each of the five services for the CILIP Cataloguing and Indexing Group conference which took place on 10th-11th September 2012. The conference has an official hashtag of #cig12, though we noticed some people are using #cig2012. Unfortunately, during the CIG conference, another event began which was using #cig2012 as their hashtag so we also collected their tweets! With the exception of TAGS (which was set up on 5th Sept) and TweetDoc (which collects tweets after the event), all the archives were set up on 9th September.

Findings

TAGS (Twitter Archiving Google Spreadsheet)

TAGS Archive

TAGS Archive (click to visit archive)

#cig12 and #cig2012 TAGS

TAGS is a Google Spreadsheet template created by Martin Hawksey at JISC CETIS. The template is available to anyone and once you open the template you just need to change the settings in the Readme/Settings tab. That will enable you to collect tweets, but if you want to schedule it to update (as you probably will for events or long term collection of tweets using a hashtag), you need to add a script to do so. The settings I usually use are to collect on a time-driven basis, daily for long term project tweets, hourly for conferences and events (there is a maximum of 1500 tweets collected per search).

TAGS will collect all tweets which meet your search criteria, which in this case included quite a bit of spam (though good for comprehensive archiving). As well as the full archive of tweets, URLs, users, who they are replying to and more, Martin has added analytics to further enhance the archive. The Summary tab shows how many tweets are in the archive, how many links, how many RTs, the date range of the tweets (first and last), and the top tweeters with their number of tweets. The Dashboard tab shows charts to demonstrate this data. There is also the option to do more advanced Social Network Analysis and to add visualisation tools.

As the result is a Google Spreadsheet, you have control over sharing of the archive. You can choose to have it private, to share with certain people, or to make public. I would recommend sharing it as view only (not edit) so that there’s no risk of someone accidentally changing the settings and tweets no longer being collected.

The one downside of TAGS is that compared to some of the other services it’s not so easy to read through the tweets (there’s a lot of data and the interface isn’t designed for reading tweets. It’s probably better as an organiser’s tool rather than one for sharing tweets with attendees/followers.

TAGS conclusion: Comprehensive search, full control of sharing, excellent organisers tool but not the best user interface for sharing tweets.

Twubs

Twubs archive

Twubs archive (click to visit archive)

#cig12 Twubs

Twubs has some neat features to help people follow hashtags. People can post to the Twubs area without having a Twitter account and doesn’t rely on Twitter (useful for if Twitter goes down). It’s therefore a useful tool to follow conferences/events and post to them so may be useful during Twitter chats. However, it only displays 10 tweets at a time which is unlikely to be enough for most conferences (unless you’re following from afar and using Twubs rather than a Twitter search). It does have an embed feature though which would be really useful for on conference web pages.

One of the handy features of Twubs is the ability to bring in additional hashtags which is particularly useful if more than one tag is being used at a conference. For the CIG conference this meant I could add #cig2012 in addition to the official hashtag of #cig12 and bring them together in one archive (in Settings > Customize Feed). You can also block users or terms which is handy for eliminating spam.

Twubs keeps data forever and updates them to build your archive, however I can’t see a way to view more than 10 at a time or download tweets.

Twubs conclusion: useful for following event (particularly for those without a Twitter account who can post if they register for a Twubs account), and handy embedding tool for conference websites. Wouldn’t recommend as a pure archiving tool.

Eventifier

Eventifier archive

Eventifier archive (click to visit archive)

#cig12 and #cig2012 Eventifier

Brian Kelly wrote an interesting blog post on Eventifier, though tested after the event so wasn’t able to pull in as much content as if you set it up in advance.

When I set up the Eventifier for #cig12 I was a little confused as to why I had to wait for the service to be set up, but having received within an hour or so what appears to be a personalised message with details of the archive including both #cig12 and #cig2012 tags (even though I had only specified the official tag) and with the CILIPCIG account tied to it, I was very impressed.

I like the additional services offered from Eventifier – it’s not just tweets that are recorded but also photos, videos and slides. It seems to have found some of the additional materials for the #cig12 conference including photos and one set of slides, though I’m not sure where these have originated from. It’s a shame for example that the photos don’t link to the original source. The tweet archiving seems to be comprehensive – it contains a similar number of tweets to the TAGS. This of course does mean that you get the associated spam, hence why many of the accounts on the Contributors page are spam accounts. Viewing tweets works well and you can also view trends and search the tweets which is a really useful feature.

It has an attractive interface and displays the different types of media appropriately. All archives are public and available only on the website – there is no way to download/export information. I find it a little strange is that there is no way to set up an account so you can’t easily find all your archives (unless you keep a note of the URLs).

Eventifier conclusion: useful tool for event amplification (i.e. sharing with attendees), though not easy for organisers to control – set up by Eventifier team and no way to amend or add additional content.

Tweet Archivist

Tweet Archivist screenshot

Tweet Archivist screenshot (click to go to archive)

#cig12 Tweet Archivist

This offers an interesting interface and some useful charts and graphs, though in order to download tweets you must pay. It also makes it clear that it may not contain all the tweets, and this is the case with the archive (though I think it has most of the ones from during the conference, just for some reason not those after the conference).

I thought it was private too, but have now figured out how to make it public. Not a totally intuitive interface – many time I clicked and it just took me to a page wanting me to subscribe.

Tweet Archivist conclusion: some interesting graphs but not much else to boast about (particularly with the free version, paid may be more beneficial).

TweetDoc

TweetDoc archive

Tweet Doc archive (click to view archive)

#cig12 TweetDoc

TweetDoc is slightly different to the other services mentioned here as it is more suited to collecting tweets after an event rather than scheduling beforehand. This is useful if you forgot to set something up, but it does have some downsides (mainly the coverage). It enables you to set a specified date range, but it has a maximum limit of 500 tweets. For most events nowadays this is unlikely to be enough, so if you were using TweetDoc you would probably need to run it periodically throughout an event in order to capture all tweets. I’m also a little unsure about its extensiveness – I asked it to collect from 10th September to 11th September (the conference dates) but the first tweet it collected was at 11:16 on 10th (and this was definitely not the first of the day) and the last tweet it collected was at 21:56 on 10th. Comparing it to the TAGS archive, it doesn’t seem to have collected every tweet from that time period – it has left out spam, but it’s also left out some useful tweets. There isn’t much in terms of added value either, it gives some of the trending words and pictures of those involved in the conversation but no statistics.

TweetDoc conclusion: Useful for after an event if you forgot to set up an archive before. If you want an idea of some tweets this might be suitable, but not for a comprehensive archive. 

Recommendations

It really depends what you want to use it for as to which I would recommend, but the ones I’ll be using in future are TAGS for organiser statistics, and Eventifier for sharing tweets with others.


Whilst writing this post I was alerted to another couple of options – too late to test it out this time but they deserve a mention.

Using Storify

Again something to be done after the event, but I like the way Storify displays tweets and it’s easy to pull in tweets from a Twitter search. You can also add additional context this way so for example you could add text to explain which session the tweets refer to, could rearrange for themes from the conference, and can add in additional information such as presentations or photos. I didn’t think to include this until I was writing up the blog post, but here is an example of some of the tweets from #cig12 presented as a Storify.

Using IFTTT (If This Then That)
EDITED 27/09/12 – IFTTT can no longer offer this unfortunately. 

As I was finalising the blog post I spotted this tweet:

I've just realised @ have added a Google Drive channel that will be really handy for creating automated spreadsheet archives. Nice one

I won’t do a full write up of this as I’ve only set it up today, but if you’d like to test it out you can use my recipe for creating a Google Spreadsheet from a Twitter search.

I’ve also set up a recipe to send tweets from a search to Storypad on Storify - this should make it easier to create a Storify after the event as won’t need to rely on Twitter search. However…

These IFTTT recipes should be good for the project tags (longer term, lower frequency) but not for events as they are limited to 15 tweets each time they search.

 

3 Pingbacks/Trackbacks