The web makes so much information available that we sometimes forget that there are still many hidden archives and collections that are not immediately accessible by way of a simple Google search.
One example is the pages created by government departments that house reports, policy papers and the gamut of related materials that are collected by the departments to keep the public informed, and which are often commissioned to inform the government of issues and concerns that may form part of policy. One of our academics was concerned recently when she went to the Department of Justice website to locate a report she had written several years ago for the previous government. The report, and all associated material, seemed to have vanished, without any indication of any new location, nor a clear lead to what had happened to it. Her assumption was that the new government, on coming to power, had removed all traces of the previous government's documents from the public sphere.
She was right that the new incumbents had taken an e-broom to the website and created a literal ground zero from their start in office.. This would not be so bad had they left an easy trail of where one could find all previous documentation, rather than creating a modern equivalent of rewriting history as if nothing had existed previously. On their website, there is no link to older material on the Research Papers page, where one would have expected to find this type of report, but when you trawl around, there is a link of sorts on Policy pages (!) – the breadcrumb trail is
Home» Publications» Policy» Ministry of Justice
… on the right hand side there is a note which says ‘Archived pages’ with a National Archives. How the average user would know this is where one might find items such as this report is questionable.
Since 2007, UK government information is harvested by the National Archives, including the entire corpus of reports, etc, produced for the DOJ under the previous Labour government. The NA is obliged under its charter to harvest government department websites on a regular basis, and keep the results online, in its site. The date when a site was last harvested can be found with some digging around, and the remit for web continuity is also explicitly stated. This will go some way to saving born digital information of an official nature, but is not fool proof, because the harvesting is not continuous, so some items might go up briefly on a department’s site and be taken down before the next harvesting period. For example, in 2012, the Ministry of Justice site was harvested on January 19, and then not again until February 27, a gap of over 5 weeks.
Interestingly – frustratingly – a Google search for the title of a report does not return reports listed on the National Archives site. An advanced title search on the National Archives own site does not return the report. You need to know the title, and that it was delivered to the Ministry of Justice, and then search in an A-Z list on the harvested MoJ site.
Sound complicated? It sure is. In many ways this one example is a symptom of a much greater long term problem for legal scholars, researchers, and historians. Because there are no longer general uniform retrieval systems in place, such as online catalogues, or even (don’t say it too loudly) the old fashioned, and not always perfect, card catalogues, the systems of organization that once provided a consistent means of locating government papers has become fragmented.
Organisations make decisions about their web presence without always planning for the future or realising the wider long term implications of their decisions. The web is not neatly indexed and organised, nor can it be.
The situation was reinforced recently when we were notified that Parliamentary Vote Office will no longer produce hard copy of the Public Bill and General Committee Proceedings in official bound volumes. They are now published on the Parliament website in html and pdf format only. The last completed paper set of bound volumes of general committee proceedings are for the 2006/07 session; no more will be produced. In the UK Parliament, the Vote Office is responsible for the provision and distribution of all UK and European parliamentary papers required by MPs in the course of their work. Bills, acts of Parliament, copies of Hansard and the Order Paper are all provided by the Vote Office. Our concern grows that the change to the publication of Proceedings may foreshadow a similar change for the Bills, Acts, Hansards and Order Papers. If we could be informed of changes, and we knew we could rely on web continuity for a full record such as we have now in our Official Papers collection going back over 400 years, we would feel more comfortable with these changes that seem to take place on such an ad hoc basis.
And one must also wonder how the ordinary punter gets to know that this is what has happened to more recent information from the government. Using one of the common search engines does not locate the report which sparked this blog piece. It requires someone to know the background developments that have taken place for storing these documents, to know what the archive is and does, and to feel competent in searching it. That is not open access, in the way that a book collection on shelves in a library is open access. The concern is that the majority of people do not use structured searching, they prefer serendipity, and they are thus likely to end up missing what they require without even realising that most government documents actually do exist somewhere on the internet.