As is often the case with over-hyped buzzwords, big datas reputation in the enterprise sector tends to be more than a little inflated. To hear some people talk about it, its the holy grail of analytics, an impossibly powerful business tool thatll put you centuries ahead of the competition. While itd be pretty awesome if that were true, reality is a touch more nuanced.
Fact is, if you simply dive in to big data without understanding how it works – and without a clear idea of what it cant do, then youre setting yourself up for significant disappointment not to mention a ton of wasted capital.
So, lets talk about that – lets see if we cant dispel some of the mysticism around big data analytics.
The Public Mystery That Is Big Data
Today, we live in a world of data, writes software engineer David Aurbach. Twenty years ago, we didnt. Just as computing power has exponentially increased over the last 50 years, doubling every two years or so, the amount of computational data has been doubling at a similar rate. 90% of all the data in human history was created in the last two years.
An impressive figure, isnt it? In the face of such statistics, its difficult not to find big data at least a little impressive; difficult not to be at least slightly wowed by it all. Maybe even a little intimidated, in light of social engineering experiments like the one Facebook carried out on its users.
That controversial experiment caused a lot of folks to look at big data in a new light. Suddenly, we began to wonder if every like, post, and click was being measured. Suddenly, we began to fret that we were being tracked. Suddenly, we began to worry that somewhere, someone was painting a complete, absolute picture of who we were as a person based solely on what we do – all without our informed consent, of course.
For his part, Aurbach isnt entirely convinced big datas such a big deal – and to drive home exactly why, he points to a similar study unveiled in July by OkCupid.
Unlike Facebook, which published its findings in a humorless academic paper, OKCupid treated its results with some serious skepticism, raising the question: What is big data actually good for? Does it even work?
Good question.
Aurbach continues on to say that, though were collecting countless petabytes of data, we arent actually doing all that much with it. Even the NSA, with its massive Utah Data Center, doesnt really have the tools to properly analyze everything. In short, most of our big data isnt even being used – and one questions if it necessarily can be used.
Understanding The Big Data Hype Cycle: Why Were Headed For The Trough
Aurbach next points to something known as the hype cycle, a term that basically outlines the path taken by every new technology as society warms up to it. Basically, the whole thing starts with an innovation trigger – something that gets everyone super excited about the new piece of tech. That trigger is then followed by ever-increasing expectations as hype reaches a fever pitch, helped along in large part by tons of people throwing around buzzwords they only partially understand.
Then, just as expectations reach a peak, suddenly everyone starts to understand what the technology really does. More accurately, they start to understand what it doesnt do. People start to grow frustrated; they wonder why theres no killer app – why hasnt this incredible, ground-breaking new tech yielded concrete results yet?
This is known as the Trough of Disillusionment – and Big Data is teetering right at the edge of it.
The bad news is that plenty of the current big data pundits are in for some pretty serious disappointment. The good news is that this disappointment will soon lead to a more complete understanding of exactly what big data is. With that understanding, the technologies surrounding this data will start to mature.
In other words, well start to see real, actionable results.
So What Can Big Data ACTUALLY Do?
Alright. Now that you folks have a decent understanding of big data – and the fact that its an overhyped technology – what can it actually do for your business? It may not be the Holy Grail of analytics, but its still extremely valuable, right?
Yes and no. The first thing you need to understand about big data is that, without the proper talent in your organization – without people who are trained in data science – youre going to have an extremely difficult time pulling anything of value from it. In addition, it isnt a cure-all; if theres something broken about your business, pulling in and analyzing a ton of big data isnt going to fix that.
If fifty years of artificial intelligence research have taught us anything, its that every problem is different, writes Gary Marcus of The New Yorker. there are no universally applicable solutions. An algorithm that is good at chess isnt going to be much help parsing sentences, and one that parses sentences isnt going to be much help playing chess. A faster computer will be better than a slower computer at both, but solving problems will often (though not always) require a fair amount of what some researchers call domain knowledgespecific information about particular problems, often gathered painstakingly by experts.
What Marcus writes there is one of the most enduring truths about data analytics – and really, about any other field. You cannot simply expect insights from big data to come to you on their own. You need to work to pull out anything worthwhile; and any data you examine needs to be looked at in context.
As for what big data can do to an organization that understands how to work with it?
One word: knowledge. At its core, thats what big data is all about. It equips you with the capacity to make more intelligent decisions, with the knowhow to streamline and optimize your business, and with the ability to make your employees work better than ever before. By studying the information gathered about your business, its competitors, its customers, and employees; you can put yourself well ahead of the competition – but only if you know exactly what youre looking for.
Go in blind, without a clear goal, and big data is little more than digital soup.
