Open policies and practices are needed to save our digital memory, says Richard Ovenden, the Bodley’s Librarian at the University of Oxford
By RICHARD OVENDEN
LAST YEAR Flickr, a photo-sharing site, announced it would cut its free storage from 1 terabyte (more than 200,000 images) to just to 1,000 items. Starting this month, many users may find that their content is in danger of being deleted. This throws open questions about the “free” business model. But more significantly, it is about digital preservation. Active users will pay or transfer their photos; others may discover that a visual record of their life is irrevocably lost.
Society has developed institutions to prevent such misfortune and act as guardians of information: the humble library and archive. My own institution, the Bodleian Libraries in Oxford, holds parchment manuscripts containing texts written almost 2,000 years ago and still perfectly legible—without any screen, battery or power cord. There is no click-through licence to ignore or password to forget.
But knowledge looks different today than it did several millennia ago. And an open society needs access to its store of records. Archival practice has its origins in the administration of governments, keeping tabs on such mundane but vital information as property records, tax and import-export details. Even in the ancient world it was recognised that access to these records was important for a healthy state administration. Today information is the lifeblood of government, business and personal life.
The growth of digital records has made preservation ever more important. For the state, it is essential to ensure democratic accountability and good governance. Yet it has become highly precarious as well.
In December 2018 the Maine state government revealed that it had suffered a catastrophic loss of public documents from the administrations of Governors Angus King and John Baldacci, stretching from 1995 to 2011. Most state-government emails sent before 2008 were irretrievably lost. Moreover, many other kinds of documents had been destroyed by state officials before they reached the state archives. Not only has information for future historians been lost, but these emails could also contain information that could be vital in legal cases.
As the work done by regulators and lawyers on the LIBOR scandal in 2015 has shown, email records when pieced together can tell a story in enough detail to secure a conviction or prevent a defendant from going to jail. And as the Brexit debate focuses ever more closely on the Irish border issue, customs records may become a defining issue in the political mess. The preservation of digital information is a cornerstone of good policy.
In some instances, digital preservation can be a life-or-death issue. Take the nuclear industry. Society will need to know long into the future—even hundreds or thousands of years hence—where nuclear waste has been stored, what it consists of, when it was deposited and so on. These data exist today, but the challenge facing the Nuclear Decommissioning Authority and other organisations is to ensure that property developers, mining companies and water suppliers, as well as local authorities, governments, regulators and the public, have guaranteed access to it in open, future-proofed formats.
So how should organisations deal with these issues? Three things are essential. The first is leadership. Organisations need to designate someone to be responsible for knowing the technical, legal and social aspects of digital preservation, and to ensure adherence to best practice. That person can act as a locus for the development of policies and strategies and exert pressure for improvements.
The second is resources. Organisations must provide funds for digital preservation. This may mean hiring staff with the right skills, experience and mindset; a place to start is digital archivists or electronic-records managers. It also means investing in technical systems and workflow processes that comply with industry standards. The business case for this is compelling. But the public sector sometimes struggles, especially when libraries, archives and museums also have thousands of years-worth of analogue materials to maintain.
At Oxford’s libraries we spend around 2-3% of our annual budget on digital preservation, twice as much as in the past, and we are reviewing whether this is enough. Other organisations should probably count on spending a similar share.
Third, and arguably most important, is collaboration. The Digital Preservation Coalition (DPC), of which I am president, supports a community around these issues, with training courses, online tools and reports. Since it was established in 2002, it has grown from ten research libraries and archives to 88 institutions including the European Central Bank, the World Health Organisation—and yes, Britain’s Nuclear Decommissioning Authority.
Not all sectors will find it easy to collaborate in this way, but joining consortia such as the DPC or the Open Preservation Foundation can provide training, information on emerging technologies, standards and tools without having to reinvent the wheel single-handedly.
At the same time novel partnerships are possible. Organisations can work with others facing similar challenges over long-term digital preservation. Despite our long-standing rivalry in boat-racing, we in Oxford work with our colleagues at Cambridge University Library on a collaborative initiative to develop common policies and procedures, create shared training tools and help our institutions on nitty-gritty issues like preservation audits. (Through ours, we found we had over 130m image files.)
These collaborative practices are well understood in the library and archive world, and the technology industry is now taking them seriously, too. New companies are forming to help organisations big and small tackle these problems. Many of these solutions are being developed through open communities. These open approaches are vital in software development, standards and tools such as JHOVE, for the important tasks of validation and format identification.
As the problem of digital preservation grows, more creative and muscular approaches are possible. Countries could impose a “memory tax” on the tech industry to fund the maintenance of digital records in the public sector. Taking their cue from banking regulation, major web platforms could be required to present their data-preservation policies in clear language when users sign up, and commit themselves to holding the files for a period of time so they can be easily transferred if the policy changes.
But for now it is librarians and archivists, the custodians of the past, that are the advance-guards of the future. They have worked with open approaches to software development, data practices and scholarly communication for years. These communities, networks and processes are a vital part of the solution. That way we can hope to avoid losing our wealth of electronic data and our collective social memory, and avert what Vint Cerf, one of the creators of the internet, fears might otherwise become a “digital Dark Age”.
_________
Richard Ovenden is the Bodley’s Librarian at the University of Oxford and the president of the Digital Preservation Coalition, dedicated to long-term access to electronic information.
