Mining the Twitterverse for Current Awareness

The nature of current awareness is changing. It has become increasingly easy for us to read newspapers from other cities, to retrieve transcripts or recordings of television and radio programs. We can monitor the progress of legislative debate without the need to read every word in Hansard. The blogosphere is a source of commentary and analysis. The “current” is more like a tidal wave than ever before.

Librarians are used to providing media monitoring and environmental scanning services. We’ve gone from clipping the local paper and photocopying tables of contents to harvesting RSS feeds, searching global newspapers on the web or through paid mega-sites. These tools have the advantage of a certain amount of stability and control. There’s a lot of information, but it’s manageable.

And then there’s Twitter. There’s been a lot of buzz on SLAW about what it is, who’s already there, how lawyers are using it or IF they should be. Librarians, heed the rallying cry — if your clients are in the Twitterverse, you need to be too.

Yes, I know. I have been publicly skeptical about the quality of the discourse possible in 140 characters, unconvinced that you can discover the content that you need to be aware of, and just generally curmudgeonly about this new toy. That’s my job. I let Connie, Steve and my other SLAW colleagues discover and explore new technologies at the bleeding edge. Once they have matured a bit, I look for practical reasons to care about them. I think we may be getting there.

I can certainly see that a mobile workforce would appreciate aspects of Twitter – it’s nimble, informal and fun. But for those of us who are predictably at the office most of the time, why bother? I’ve spent the last six months or so quietly fiddling with Twitter in different environments, and I think there may be something worth knowing here, even if you aren’t of the two-thumbs typing demographic (that is, Blackberry users).

The tipping point came for me when I heard that the Ministry of Labour and Ontario Ombudsman had accounts. I’ve also started to see bloggers create Twitter feeds for their posts ( HYPERLINK “” ComplexD, for example, and SLAW, of course) – this, I love – you can actually get a little bit of information about the post, so you can decide whether or not to go to the blog. Twitter Search is proving to be a valuable addition to the toolkit as well.

A couple of months ago, I had Tweetdeck loaded onto my work computer. Tweetdeck allows you to manage the flood of Twitter traffic. It breaks the Twitter stream into columns, which you create by writing searches. Several useful applications (such as a URL shortener similar to TinyURL) are available in a toolbar at the top of the display.

I’ve had it on my home pc for somewhat longer, but since I actually live on my work computer, I felt it would be a more useful experiment to try it there. My selection of Tweetdeck as the tool of choice for managing and filtering feeds was pretty random — I’d heard of it, and IT was willing to load it. My interest is in the possibility of usefully filtering the Twitterverse, not evaluating the filters themselves.

I’ve created a short slide deck [also on SlideShare] showing some screen captures of my work account. I have created two — a personal one and one for work, which I may share with the library staff. The work account is a lurker — I don’t ever plan to tweet from it. If you’ve never seen Tweetdeck, you might want to refer to the screen shots to better understand my comments.

The aim with any filter is to show you the useful chatter and minimize the noise. Filtering in Tweetdeck seems quite simple at the outset, but does take some getting used to. Each filter is presented as a column in what is essentially a portal. The first column you’ll see is the feed from everyone you’re following. By default, you also get a column for replies and direct messages. After that, each column reflects a particular search statement you’ve written. The left-right scrollbar is an essential tool here, as new searches are plunked to the right of your last column – you can move it to the left (and into view) by using a button at the bottom of the column. Other buttons in this area allow you to mark tweets as read, clear the column and clear only the tweets marked as read. Note that the results you see are from the Twitterverse as a whole — not from people you have specifically selected to follow.

As your list of those you are following becomes larger, the Groups functionality may become useful — it may also be necessary, should you create a large number of filters (it appears that you can have 10 columns, but no more). It allows me distinguish my librarian feeds from my government feeds, for example. This will allow me to review in a glance all tweets from particular sources, rather than the topical arrangement available through the search-based filters.

The toolbar at the top of the Tweetdeck screen includes a number of fun toys. You can get a feed of stock information (not in this market, thank you!). Just click the $ to activate StockTwits. Want to see the current buzz? Twitscoop creates a column with a cloud display of the most frequently occurring terms in the twitstream.

While I don’t pretend that I have all of the answers on effective filtering of the Twitterverse, I have learned a few things:

• Tweetdeck will not allow you to edit searches, so you’re best served to create the search logic in Twitter search, and the cut and paste into Tweetdeck. If you have made a mistake, or if the need to filter on that particular search goes away, just remove the column from the display.
• There is no controlled vocabulary in Twitter, so be creative. Use as many synonyms as you can think of. Try to anticipate abbreviations of your terms – 140 characters is not much space, so how likely is a tweet to include a term like “workers’ compensation insurance appeals tribunal”? Really.
• Think about what your clients are going to want to know from the Twitterverse. Firms may want to know what is being said about clients or about themselves. Government departments / ministries may want to know about reactions to policies, legislative changes or current events. Remember that there is no thesaurus, tagging or cross referencing, so if you want to see “everything” about a target, use lots of terms and OR in your search logic.
• Start broad, and be prepared to refine the search over time. I have several very specific searches created to discover comments about my employers, and I’m very happy that they have never activated. I also have a feed for the word “Ontario” which is endlessly busy. I may not read every tweet that results here, but I have discovered some interesting information (like the Ministry of Labour account and the Ontario Ombudsman).
• Don’t expect too much. The tweets I value right now usually include a shortened URL to a blog post. However, for those of you watching Twitter for reputation monitoring, every mention may have value.

These are early days for me in the Twitterverse (Twittosphere?). I would be very happy to hear from others who filter it for their own uses or as part of a current awareness or competitive intelligence program. What are your best practices?