The Rise and Fall of Big Data Hype

We are still a few years away from fully utilizing big data, but we have now moved past all the big data hype.

Every year, Gartner releases their Hype Cycle for Emerging Technologies, and in 2014, we can see that big data hype is on its way out, and we can now start to focus on its practical applications, rather than focusing only on the broad concept itself.

Source: Gartner

We can clearly see that there are five stages of response to emerging technologies, and when we look at where things like the Internet of Things (right on top of the Peak of Inflated Expectations) and Speech Recognition fall (in the Plateau of Productivity), we start to get an understanding of what these different categories are.

Essentially, in the lifecycle of an emerging technology, use cases, excitement surrounding and responses to a product or concept vary considerably based on where the product or concept falls in the hype cycle. It is once it leaves the Peak of Inflated Expectations that we can begin to identify its real, valuable applications and it can be applied to a variety of markets at scale.

Where was ‘Big Data’ before?

For a long time, we have found big data in the Peak. Some examples of this are sites like Spurious Correlations, and Google Flu Trends. There is a lot of hype, but little value pulled from the applications. Moreover, there is little proof of concept.

With regards to business applications, the concept of big data was shrouded in hype and preached with excitement, but there wasn’t much of a business case made for quite some time. We knew that the vast amounts of data that were being collected could be beneficial to our operations, but for the time being it was simply unstructured data that could be used in a few tests and unproven methods.

Slowly, people started to realize that, while the applications could be developed, they hadn’t yet been perfected.

What is meant by ‘Disillusionment’?

As we can see in the chart above, big data is headed in to the Trough of Disillusionment. What exactly does that mean? Will we see the abandonment of big data by some brands, or even entire industries?

Not quite.

Essentially, as people begin to get somewhat of a grasp on big data – at least conceptually – they are beginning to feel a degree of disappointment in what it can currently offer. There are quite a few companies that have found unique, creative and valuable ways to use these data sets, but for the majority of brands (even the majority of industries) this has not yet been the case.

Currently, the big data hype that we have seen surround the fields of marketing, sales and operations is dying down. People are realizing that there is no magical solution that will be made clear simply by “using” big data. Data sets are tools, and brands are starting to realize that in order to use these tools, they need to become intimately familiar with their inner workings within their own brands and industries.

What is coming next?

Though the big data hype seen over the last few years may be quieting down, it does not mean that the applications surrounding big data will be abandoned. There are still plenty of startups and established companies that are looking for new and useful ways to use these data so that we eventually find ourselves in the Plateau of Productivity.

Now these data are being pulled and used for things like predictive analytics, business intelligence and other exciting emerging fields. The key for brands now is to think about data on a micro scale.

It might sound somewhat paradoxical, but in order for brands of all sizes to start taking advantage of their data, they will need to approach it on a smaller scale. When large brands have hundreds of terabytes if not petabytes of data, they can afford to run comprehensive tests because there are proofs that can be found within these data. For smaller brands that might not have all that information, they need to think about their information somewhat differently.

Looking at smaller data sets that are specific to a given initiative, social network or ad campaign can provide plenty of valuable information ranging from targeted audience, to high-performing campaign types to rhetoric on social media that outperforms the competition.

Thinking about data in this way will help move us through move us through to the Slope of Enlightenment, and eventually allow all brands to be as productive as possible using their data.