A Human Perspective on the Power of Big Data

By the time 2020 hits, the international business community will be holding a gigantic hoard of data. According to research from Druva on the growth of corporate data, by the end of this decade the volume of corporate data worldwide will hit 10.5 ZB, up from 2.73 ZB in 2015. One zettabyte equals a trillion gigabytes.

We know the saying Knowledge is power from countless repetitions of the phrase (Wikipedia and other sources say it originated from Sir Francis Bacon). As substantiation of our belief in the truth of this maxim, weve seen the rise of big data and weve seen governments collect data on citizens in the attempt to turn the numbers into knowledge, or intelligence. Then, if we believe the maxim, the knowledge generated by data equals power. Its not hard to see why the amount of worldwide corporate data will skyrocket 8 zettabytes in 5 years.

Sociologist Robert Staughton Lynd said, Knowledge is power only if man knows what facts not to bother with. So, in order to make big data smart data, weve got to know how to separate the wheat from the chaff. Weve got to understand how to manage the data.  

What we Analyze

MITs Technology Review reports 99.5% of data goes unanalyzed, and that percentage is getting bigger as the amount of data collected grows. But how do we use the .5%–give or take a few percentage points–of data to create real solutions?

The answer comes down to questions. Asking the right questions of big data can yield the type of results that really help people. Case in point: in an interview with CXO Talk, Harvard medical professor Dr. John Halamka explains how big data analysis helped cure his wifes cancer.

By asking the computer what type of treatment ten thousand Asian females with a similar type of tumor received, and what the outcome of the treatment was, Halamka was able to effectively treat his wifes cancer. As opposed to taking medical advice from a single, trusted source, Halamka accessed advice from ten thousand trusted sources. As we move into an age where this type of data is accessible, more and more doctors will be expected to use it to the advantage of their patients, while simultaneously respecting patient HIPAA rights.

The Importance of Privacy

The importance of HIPAA rights brings up a word attached to controversy in the big data age: privacy. Any discussion of big data sees privacy come up as a concernunless the discussion is more concerned with capital than ethics. Corporate decision-makers ask questions about ROIs and the bottom line, but they are also faced with the tough question about how far they should go in collecting and analyzing data on their customers. Oftentimes, this question is ignored.

Part of the reason why the privacy question gets buried is that there are so many considerations for businesses mining data. Decision-makers want to know what the trends are, and analysts need to be up-to-date on tools to communicate trends (such as data “viz”) in a way people who arent necessarily analytics experts will understand. On top of that, businesses must stay informed on the new trends in analyticsthey have to stay informed not only on consumer trends, but on industry trends as well. Thats a lot of trend to keep up with.

In his article on big data and analytics trends for 2015, Timo Elliot points out that privacy is one of the big trends, and that privacy laws and processes lag far behind the power of available technology.

Approaching this trend from a user standpoint, social media users dont exactly feel privacy is trending. But they are worried about what can happen to photos they share on networks. The user may not realize the geotags from photos are more likely to be mined for data to paint a picture of the consumers lifestyle than pirated by a porn site and used for ads. 

Approaching this trend from a business standpoint, one might ask how to use the increasing demands for privacy to improve ROI. Companies who are most able to impress upon the consumer a high level of care for privacy, and who are most able to prove they care, are the ones who will emerge on top.

Companies need to be completely transparent with consumers about what data they will collect, and what they will use it for. When a company respects consumer intelligence, and offers security combined with transparency, the privacy trend will work for the company in terms of the bottom line.

The Cusp of Greatness       

If we use data for good, and dont misuse it, we will be that much closer to greatness. Big data gives us the opportunity to look out over years and years of activity from millions and billions of people, each with a unique story, to find answers to the important questions. So lets start asking how we can use big data to find out what people need, and lets start asking how we can use big data to decrease clutter and waste. If we ask the big questions, the big answers will materialize.