What Can Be Done to Better Manage Big Data in Healthcare?

Nik Stanbridge, Arkivum

The UK healthcare sector, and particularly the NHS, has changed and reformed more in recent years than ever before. In the face of significantly reduced budgets and huge demands to reduce costs and increase efficiency, this has prompted radical cuts and the introduction of new technologies in order to reshape the industry from top to bottom. In fact, technology is seen as an enabler of change and is being adopted in a wide range of areas.

Today, in order to meet the demands for reduced costs and increased efficiency, many NHS Trusts are implementing new digital strategies and, as a result, this is generating vast volumes of data. Big Data is therefore driving the need for new and innovative digital workflows, which is fostering new ways of working and transforming how patient care is delivered. As well as reduced costs, the value of actively managing Big Data in healthcare also delivers a number of other benefits, including compliance with regulatory requirements, eliminating backup concerns and ensuring data is secure for the long term.

However, as the data generated by today’s pioneering work in the sector continues to grow, many healthcare organisations, including hospitals and research institutions, are finding that storing this vast amount of data in a way that fulfils a range of needs and requirements is much easier said than done.

Key criteria that organisations need to consider include:

• How long the data is going to be kept for
• Whether the format that the data is being stored in will be readable when it is retrieved in the future
• At what rate the data is going to grow
• Meeting compliance of the NHS Information Governance (IG) Toolkit
• Ensuring the highest standards of data security via the approved N3 network

As an example, digital pathology is one such area that is significantly benefiting from the application of Big Data, but equally, it is grappling with the issue of finding a suitable home for the sheer quantity of data being created.

Traditionally, pathology involves examining tissue slides under a microscope, however many NHS pathology labs are now embracing digital workflows, which involves converting the glass slides into digital images that can be viewed, managed and analysed in a way that transforms digital pathology practices. It is widely believed that digital pathology has significant potential in achieving quicker and cheaper diagnoses and prognoses of life-threatening diseases such as cancer, and the field is seeing significant growth as a result.

Nonetheless, to put into perspective how much of a challenge storing the data generated by digital pathology processes can be, a small slide scanner running 200 slides per day at medium resolution will generate over 20TB of data per year, which has to be stored, managed and kept secure over decade-long timescales, all the while taking into account the compliance, security, cost and data integrity requirements associated with its storage.

Looking at numbers like these, and when you start to consider what is required to successfully store this data, it quickly becomes clear that digital pathology laboratories, as well as hospitals and various other medical research institutions, have a significant task on their hands. The big question being asked by the majority of specialists who are trying to get to grips with their long-term data storage strategies is, “where do I start?”

There is a demand on NHS IM&T (Information Management & Technology) Managers to solve this problem, some of whom are expanding their local infrastructure while others are implementing cloud-based services such as Amazon Glacier or Google Nearline. However, simply expanding the local infrastructure to provide sufficient space for these data volumes does nothing to ensure that it is protected from corruption or loss.

The good news is that there are specialist, managed data storage services which are positively disrupting how Big Data is being preserved by reducing costs, meeting NHS and healthcare compliance requirements and delivering the long-term efficiency benefits that enable digital workflows to flourish. Therefore, IM&T Managers should consider bringing in one of these specialist providers of long-term data archiving, such as Arkivum, who can implement a managed service that has been specifically designed from the ground up to provide ultra-secure storage for large volumes of data for extended periods of time.

Such specialist services also help IM&T managers solve their backup problem. In essence, if data is static and no longer likely to change (slide images being a case in point) then this data can be offloaded to much cheaper, and more secure, long-term archive storage. By doing this, IT managers can economise by not needing to provide extensive and expensive infrastructure to maintain an unnecessarily large backup window. While data volumes are very large, using a managed data storage service as described above does help to eliminate many IM&T Management headaches. Predictable costs and long-term contracts, along with significant local IT cost savings, also provide a very compelling business case.

Healthcare organisations across the board are looking for ways to preserve their data and I would strongly encourage hospitals, genetics laboratories, fertility clinics and digital pathologists alike to consider the integrity, security and longevity offered by a digital archiving solution, as it is by far the most effective way to ensure that data is protected.

If you are interested in finding out more, please download a copy of our whitepaper: Managing Big Data – Reaping the Rewards. In this paper, we detail the issues and challenges that are being faced by healthcare organisations and in particular digital pathology labs when it comes to managing large volumes of data and we highlight what a digital data storage strategy needs to include. Alternatively why not visit the Arkivum team on Stand 22 at the UK E-Health Week Show on 19th and 20th April, at Olympia London.

About Arkivum

Arkivum is an innovative, fast growing data archiving provider, specialising in delivering long-term archiving solutions with unique bit level preservation and an unrivalled 100% data integrity guarantee. Formed as a spin-out from the University of Southampton IT Innovation Centre, Arkivum’s technology experience and expertise is fundamental in enabling it to ensure core solutions meet the needs of its customers.

Since its formation in 2011, Arkivum has grown steadily, boasting 300% growth in 2014 and again in 2015. Arkivum provides archiving solutions to 1 in 2 of the top 60 universities in the UK, but its business growth is not only within the higher education sector. The need to retain and provide access to large volumes of data for many years to come is also paramount in other sectors too, such as healthcare, life sciences, heritage and culture. For more information, visit www.arkivum.com