Don't be afraid of the dark (data).

As data and information specialists, we’ve seen a fair few scaremongering campaigns about dark data in our time. It’s true that dark data can cause problems for your business, but what makes it such a common topic for sparking fear among data and information leaders? And is there really anything to be concerned about when it comes to dark data?

What is dark data?

Put simply, dark data describes the information that an organisation collects and stores, but doesn’t use for other purposes, like data analytics, to actively drive insight or aid decision-making. That’s how research and advisory firm Gartner defines dark data.

Think about it like this – do you ever shove things to the back of the wardrobe, thinking “Maybe I will wear those boots one day” or “Maybe that leather jacket will be worth something in a few years’ time”? Now imagine storing boxes full of those things, and you soon start to lose track of everything you’ve got in there. There’s a method to our madness when we store things we just don’t use or need. But, usually, we just end up with an overflowing wardrobe, full of things that haven’t seen the light of day in years.

Though occasionally it’s the perfect place for finding treasure you didn’t know you had, a messy wardrobe probably isn’t a metaphor you want applied to your enterprise information stores.

The three main challenges of dark data

So, what’s the problem with keeping things piled in the back of your metaphorical data wardrobe?

Firstly, storing more data costs you more money. If we had to pay for every hanger space in our wardrobes, we’d probably be a bit more ruthless about what we kept. The same applies to data too – why hang onto it if your organisation isn’t deriving value from it or doesn’t need to keep it for compliance purposes?

So, how do you know whether there’s insight to be found, or if a document needs to be held onto? An information management and discovery platform like Insight Engine is one way.

Secondly, there’s likely to be value lurking in lots of your dark data, you just haven’t got the tools to find it. For most organisations, only around 15% of all stored information is understood and classified as ‘business critical’.[1] That number could go up, if only information leaders had visibility into the currently unclassified, dark data in their business. True value comes from making connections between known and unknown pieces of information in a specific context, providing insight that would otherwise have been missed.

This also applies to understanding the relationships between data and people. For instance, identifying owners and subject matter experts, based on the content and types of information an individual handles or creates, can provide useful insight for knowledge workers. Furthermore, when searching for information, knowledge workers only find what they’re looking for 56% of the time. If huge areas of dark data were uncovered, indexed, and made searchable, that percentage could rise in your organisation.

There’s also another aspect to dark data; one where it’s not just failing to deliver value for the business, but where it also poses a genuine risk.

Dark data represents around 52% of stored information for most organisations [2] – that’s a hefty chunk of unknown entities. Not only are organisations often paying cloud or other storage providers to hold onto data that has no known value, they’re also storing potentially sensitive information in unsecure repositories, with no idea of the risks it could pose to the business. Personally Identifiable Information, payment card information, sensitive information in emails and scanned documents – all of these could be found within an organisation’s dark data, and all could pose a risk if targeted by cybercriminals or accidentally leaked. Achieving compliance by meeting regulations like the GDPR doesn't just apply to the structured, well-known repositories in a business – we also have to look more closely at the information we can’t immediately see.

So, should information leaders be afraid of dark data? Put simply, no – not when two of the three main challenges around dark data are in fact clear opportunities for reducing costs and driving more value or better insight from information. As for the third challenge, information risk, we’re only afraid of our dark data because we can’t see what's hiding. As soon as we reveal what’s there, there’s nothing to fear, just a cupboard to clear out.

With the fear of dark data behind you, having a better understanding of your organisation’s information enables you to embark on transformational journeys, consider smarter architectures, and adopt cloud or hybrid models more effectively. What’s really scary is missing out on the opportunities that come with changing how your organisation thinks about data dark.

Start automating your data governance today. Book your free no-obligation Data Risk Assessment.

[1] [2] The Databerg Report, Veritas 2016