Every tweet you make, they’ll be watching you

It was appropriate that the Library of Congress twitterfeed (@librarycongress) broke the news last week that the LOC would be acquiring the full archive of Twitter, receiving billions of public tweets from the 2006 inception of the service to the present.

“The Twitter digital archive has extraordinary potential for research into our contemporary way of life,” said Librarian of Congress James H. Billington in a press release. “This information provides detailed evidence about how technology based social networks form and evolve over time. The collection also documents a remarkable range of social trends. Anyone who wants to understand how an ever-broadening public is using social media to engage in an ongoing debate regarding social and cultural issues will have need of this material.”

The importance of the acquisition was questioned by some, however, with a robust discussion over at GovLoop kicked off by member Harlan Wax who asked the broad question “really, to what end?” in terms of why tax dollars should be spent on archiving the Twitter content.

Twitter also blogged about their donation to the LOC (yes, access was donated, not purchased by the LOC):

Over the years, tweets have become part of significant global events around the world—from historic elections to devastating disasters.

It is our pleasure to donate access to the entire archive of public Tweets to the Library of Congress for preservation and research. It’s very exciting that tweets are becoming part of history. It should be noted that there are some specifics regarding this arrangement. Only after a six-month delay can the Tweets be used for internal library use, for non-commercial research, public display by the library itself, and preservation.

The open exchange of information can have a positive global impact. This is something we firmly believe and it has driven many of our decisions regarding openness.

Obviously they are tooting their own horn, but I think there are some good points about why this content is appropriate for the LOC to archive, if for no other reason that we are preserving a snapshot of what communications was like in the decade. There are also a number of messages/tweets that arguably hold historic value, some examples of which are mentioned in the LOC’s blog post “How Tweet it Is” (and I’ve pointed out others in the past).

A couple of questions come to mind, however, as I was reading news about the LOC’s effort. First, I will point out that once again we have Twitter working with and featuring a government agency on their official sites, yet they haven’t taken the time to add a “verified” icon to the @LibraryCongress Twitter account (see Twitter promotes case study on how government uses its network for another example of this that was later fixed). As someone who runs a directory of government Twitter accounts, I’d really like to see greater emphasis on verifying these accounts in a more official manner (heck, if it’s too big of a job for the ~130 or so Twitter employees, I bet I could convince my firm, Deloitte’s public sector, to help out with a process or pro bono project to streamline verification of government accounts).

Second, I wonder if there will there be any agencies looking to the LOC archive as a means to meet records retention policies? If the federal government is preserving all tweets, is that sufficient for an agency, or will individual agencies still want to develop a more robust method to archive their own Twitter interactions?

This post was written by
  • http://twitter.com/ScottHorvath Scott Horvath

    I had that same question this morning. If the LOC is archiving tweets then why should Government continue to do it on their own using a variety of software packages and services which don’t all archive the same way?

    LOC should build a public window into the tweets. Plus, Google has access to the full Twitter stream as well for their search integration and eventually will be showing the full stream in their “Updates” search option. If Google would implement the same Twitter search operators (eg – from: to:, etc) then there would be a public facing window to all the past tweets. Not to mention this would also help cut down on FOIA requests in the event that one is being sought for “tweets” from an agency. If it’s available publicly through Google then there’s nothing to ask for…it’s there!

    In fact if Google did implement search operators from Twitter then Twitter could simply interface with Google and finally have a search tool that works.

  • Peggy

    Hi Steve,
    I don’t know whether agencies will rely on the LoC twitter archive for their own purposes, but a recent blog post by the Archivist of the US may shed some light:
    “You might wonder why the National Archives did not acquire the tweets. Our primary purpose is to acquire, preserve, and make available for research the most valuable records of the Federal Government. Because tweets aren’t government records (although tweets of federal agencies can be), the Twitter archive is much better served by the Library of Congress as a cultural institution. At the National Archives, we are working with over 250 Federal agencies and their components to identify and schedule Federal records, some of these most certainly are tweets. Our records appraisal process identifies those records that are valuable enough to be permanently preserved.” (http://blogs.archives.gov/aotus/?p=172)
    This post from NARA’s Records Express blog provides similar information: http://blogs.archives.gov/records-express/?p=619
    Both posts say: “This summer, we will issue a NARA Bulletin that will give Federal agencies guidance about their use of Twitter and other “web 2.0” services.”

  • http://GovTwit.com Steve Lunceford

    Thanks Peggy for the info and link. Interesting.

    Scott, great minds think alike 😉

  • http://www.govloop.com Andrew

    Hi Steve – Great post. I just shared a link to it over in a blog I posted this morning over on GovLoop (“You’ve Got All Our Tweets, Library of Congress. So What? Now What?”). There are a few relevant comments that address your questions, including these thoughts from Lisa Haralampus (now at State Department, formerly at the National Archives and Record Administration):

    “For Feds, I think this has a profound implication on our Federal records management responsiblities. I think that we can draft schedules that say the tweets from our agencies are indeed records – but not records that are “appropriate for preservation” because they are already being preserved by the LoC. I see that as a cost-saving issue for government.”

  • Pingback: Social Media in the News – Guidelines for State and Local Government « History For All the People

  • http://twitter.com/AndrewPWilson Andrew


    Good post. One thing that I would add is that in many cases (and not only with Twitter but for many other 3rd party platforms) is that responsiveness to government requests (be it for “verified” status or something else) may not get resolution as quickly as would be preferable. There are valid reasons for this that everyone working in this areas should keep in mind, including: 1) These are largely “free” services and the level of service may be reflective of this; 2) federal, state and local governments are generating an enormous number of accounts on these platforms right now and, barring an issue with urgency or significant impact, it is not realistic for these platforms to respond in a timely manner to every request.

    Certainly there are mechanisms and approaches that can be taken to help mitigate these issues but, the idea of “loss of control” applies not only to message when using social media but also to how responsive we can expect these platforms to be to the needs of government.

    • http://GovTwit.com Steve Lunceford

      Agree that it’s somewhat unreasonable to expect these small private firms to tackle “verification” with the rapid adoption of these platforms by government agencies at all levels. However, when they work directly with an agency (such as the Library of Congress or earlier with USGS), don’t think it’s too much to ask that they take that extra step.

  • http://twitter.com/jessewilkins Jesse Wilkins

    I think the question of whether Twitter should default to verifying government accounts is interesting in the sense that it would be a de facto different treatment of government vs. other types of accounts. In other words, we wouldn’t expect them to verify every account for a private sector firm, much less *all* the private sector firms.

    It’s a timely thought because I just read Neal Stephenson’s Snow Crash again – one of the definitive “cyberspace” novels from c. 20 years ago. There’s a scene in it where the hero, Hiro, wants to rent a boat and is denied. He ultimately gains passage and then finds out the Mafia had originally chartered it and started discussing it with the Mafioso on board. He noted that he understood why the boat’s operator wouldn’t re-charter it to him, breaking its contract with the Mafia. The Mafioso responded to the effect that money was money and everyone’s gotta make a living. The point there was that the Mafia was just another group; the point I am making is that so is the government. You could argue whether the President’s account should be verified, considering how few of the Tweets are actually his; but I don’t see a reason for a private sector organization to put itself through the wringer to verify probably hundreds of thousands or more accounts AND then have to do it again every 2, or 4, or etc. years.

  • http://twitter.com/ScottHorvath Scott Horvath

    @Andrew: I think what I’m hearing from that quote by Lisa is that it would make sense to say that agencies do not have to capture their “publicly available” tweets since LoC already has them…but that direct messages (which are not publicly available) would need to be captured and preserved in a predetermined format (XML for example).

    If that ends up being the case, it’ll most certainly make it cost beneficial to agencies since the number of direct messages is probably much lower than the public ones.

    I’m looking forward to reading the NARA bulletin on social media records management in a few months. Hopefully, Feds will finally have an answer as to how and what to address as records.

  • Pingback: GovTwit’s 10 top gov 2.0 stories for 2010 | GovTwit