Welcome to the Post-Big Data Era

Several years ago the industry coined the phrase “big data” and we discussed what this new term meant for Dell EMC. We framed our thought process using the three Vs: velocity, volume, and variety. Taming the three Vs meant significant business insights and dramatically improved financial results for our customers.

But looking back, it hasn’t worked out quite that way. We are now living in a post-big data era where we are dealing with an increased compute capacity and massive data sets.

Over the years data took the steering wheel while compute sat in the back seat. What has been missing over the last few years (as John Roese points out in his 2018 predictions blog) is the reality of Artificial Intelligence and Machine Learning. Our massive data sets are being processed by new systems that not only need to learn and reason with huge data sets, but also need to do that in a quick and reliable way at the speed of business.

Extracting the expected insights and business value from all of that data is a challenge – and an opportunity – for organizations. John Roese mentions that Big Data will become Big Intelligence; we need to embrace “Data Valuation with Big Intelligence”. We’ve moved beyond simply Big Data.

Why Data Valuation?

• “Data”: the Dell EMC journey continues to feature the handling of mission-critical data in all its forms, from mission-critical databases to unstructured data stores.
• “Valuation”: the process of calculating or reasoning about data’s value is a matter of computing intensity.

To process this critical business information in the age of data valuation we are going to need to process data differently. This requires the combination of storage and compute innovation. It’s a good thing we’ve already been at this for a few years!

In 2014, Dell EMC launched a data valuation research partnership with Dr. Jim Short of the San Diego Supercomputer Center. Our research findings, published last year in MIT’s Sloan Management Review, highlight billion-dollar data valuation examples:

• The most valuable asset in Caesar’s Palace bankruptcy filing is the Total Rewards Customer Loyalty database. It has been valued at one billion dollars by creditors.
• LinkedIn’s acquisition of Lynda.com was mainly a data valuation exercise that also exceeded one billion dollars.
• Tesco placed an internal valuation of over one billion dollars on their Dunnhumby data asset, which contained the shopping habits of some 770 million shoppers (Kroger purchased the data for less than one billion).

The key to performing this type of data valuation will be to continually reason and value data at speed.

The “brains” of the IT infrastructure will evolve to quickly and efficiently recognize, analyze and label data, know what data goes where, identify how it needs to be stored and accessed in the future, and decide where it needs to live specifically. – John Roese

Data valuation will require the combination of multiple forms of data: legacy mission-critical data, recently-collected Big Data, and emerging forms of IoT data. Combining all three types of data together is crucial: they each represent evolving patterns of business activity over time. John Roese explains the transition from mission critical to Big Data (second wave) to IoT:

All three types of data must exist in a way that enables compute-intensive valuation. This valuation must extend from the cloud to the edge, and in future years to gateway devices.

The “Age of Data Valuation” will also require additional innovations in the areas of data trust (e.g., blockchain) and data visualization (e.g., AR/VR).

In future posts, I will expand on these technologies and their relation to data valuation.