The power of sharing knowledge in digital archiving

Learn how eArchiving helped the National Archives of Estonia gain years' worth of technical work, and how you can achieve the same.

@ Photo by NAE

Noora, the main building of the National Archives of Estonia, stores and preserves millions of records, digital images, film recordings, photos, maps, seals and much more.

Quick facts:

Meet the NAE

The National Archives of Estonia (NAE) is the government agency responsible for archival administration, active since 1999. The NAE preserves records documenting the history, culture, nationhood and social conditions in Estonia independent of the time or place of creation, or data medium1. NAE holds about nine million records provided by hundreds of public and private sector organisations, non-profit NGOs and even nationally important individual persons. All content held by NAE is accessible to the general public at NAE reading rooms, including an online reading room for both born-digital and digitised content.

A need for knowledge sharing

Archives exist to store all kinds of data in different formats. We think of them as dark and stuffy rooms, but in reality, archives store legally, culturally and historically important artefacts and national treasures of both the past and present. Estonia started digital archiving in the mid-1990s, and by 2010, the NAE was well on its way to digitally preserving national data. Archiving, however, is a complex process needing to reconcile the compatibility of past and current technologies with future solutions. It needs constant maintenance, integrations and migrations to keep data safe and accessible for generations to come. What makes digital archiving even more challenging, is that while it is legally obligatory, funding for digital archiving is often limited. This is caused by digital archiving often being perceived as a back-office task that isn’t profit generating, even though the importance of archiving goes far deeper than money. 

Since digital archiving is relatively new and expertise and experience is scarce, NAE realised that it would greatly benefit everyone if knowledge and best practices could be shared between the different EU Member States’ archiving initiatives. In 2013, the NAE formed an unofficial consortium for knowledge sharing, and in 2014, the consortium received funding from the European Commission. The consortium, called E-ARK, was a multinational big data research project that ran until 2017, and ended up paving the path to what the eArchiving Building Block is today.

Cross-border collaboration

The key concept behind the European level collaboration in archiving was based on the fact that every country has important digital data that needs to be kept for decades or even centuries. The information that needs archiving wildly varies from birth and death certificates to property ownership records. Legislation and requirements vary from country to country, but the file formats are still the same everywhere. So why not work together to find the best way to preserve these digital bits?

With the funding received from the European Commission, the consortium organised events to reach archiving experts from all corners of Europe. They shared information, lessons learned, discussed newest technologies and started building a virtual knowledge base. They also produced technical specifications and software components that are publicly available to any European organisation, public or private. These technical specifications form the European standards in digital archiving, ensuring that the solutions are compatible with each other now, and in the future. For the sake of efficiency, some Member States’ national archives have now specialised in certain areas in digital archiving, such as the preservation of geodata or relational databases. Knowledge is shared with other members of the community and everyone knows who to contact for which types of archiving questions. NAE, as the founding member, was an integral part of making this collaboration come to life.

Results and benefits 

As a result of this collaboration, NAE went from having five local digital archiving experts to a community of 50 digital archiving experts to talk with and learn from. The international archiving community consisted of a wide range of expertise from national archives, but also universities, software developers, individual experts and so on. “It has been impressive how practical and pragmatic the technical solutions discussed and shared have been. There’s something to learn from everyone,” says Mr. Kuldar Aas, Deputy Director of the National Archives of Estonia. He continues, However, what’s even more remarkable than the tangible outputs of any technical specifications and tools produced, is the collective wisdom generated and shared among participants.” eArchiving allows archivists to tackle future challenges together.

Collaboration has allowed NAE to move away from in-house custom development and raised the technical readiness for reusing common specifications and tools developed by others. This will bring down the costs of running NAE. By now, NAE has already implemented the Portuguese Database Preservation Toolkit into their workflows, and is also gradually moving towards eArchiving solutions in other areas.

According to an estimate performed at the end of 2017, the E-ARK collaboration allowed NAE to gain around 2-3 years’ worth of technical work. In other words, it would have taken the NAE years to rise to the same level of technical excellence and maturity to achieve the same on their own.

What's next 

Digital archiving never stops. It evolves continuously and NAE will continue developing and optimising its systems. While most public entities are aware of eArchiving needs, private companies are often still unaware of their implications. This could soon bring about an 'e-archiving awakening' in the private sector. NAE is already helping private entities in Estonia archive digitally, and will continue doing so in the future.

Mr. Kuldas Aas hopes that the support provided by eArchiving will also encourage and help private sector service providers develop solutions in digital archiving. At the moment there are too few solutions available in the market. “In the future, all digital archiving platforms and services will be interoperable by default. This means that they will be technically so similar that users can without any problems send files to or between any archiving systems. This is what eArchiving is helping to achieve”.

Quick tips for making a killer archiving plan

  • Get to know your data, your systems and the laws regulating archiving
  • Legal archiving requirements greatly depend on industry and policy area
  • Understand archiving needs. In business, archiving is driven by legislation, but don't forget that motivators can also be sentimental or cultural in nature.
  • Define the retention time for your data, the time span you have to keep data safe, preserved and accessible
  • Save costs by archiving only what you really need, too many organisations keep too much unnecessary information
  • Use the eArchiving Building Block and get involved in the Community

The eArchiving Building Block was established in 2018 to ensure future collaboration and availability of E-ARK's outputs. The specifications and tools produced so far are described on our website and available to all. We would be happy to help you reap the same benefits as the NAE. Visit us at eArchiving to learn more.

eArchiving on CEF Digital