Big data is no longer some nascent trend riding a cycle of media hype.
It’s here to stay, and it’s transforming how businesses make decisions, build products, and engage customers. In fact, the ability to distill mountains of data into actionable insights has become a sustainable competitive advantage, separating the Netflixes from the Blockbusters in every industry.
But you knew that, right? Most industries are well-acquainted with big data’s disruptive power. What’s news to most is the fact we’ve neglected big data’s brother, little data.
In so doing, we’ve diluted the value big data offers our businesses. Fundamentally, big data isn’t about the amount of data captured. It’s not even about the new types of data you can collect. It’s about turning transaction-level information “ which represents the most accurate observations of what’s really happening on the ground “ into business insights to support decision making.
In this way, little data is the key to unlocking big data’s true potential. Granular data can be aggregated, shaped, and molded to answer any business question, and to reveal what’s truly driving your results.
Now, you might be wondering, if big data is really about little data, why don’t we talk about it in these terms? Why don’t we associate big data with its granular constituents? Well, one reason is the analytics community’s current mainstay, business intelligence or BI, can’t scale to provide comprehensive, enterprise-wide insights. Instead, BI relies on visualizations created from isolated datasets and static dashboards. Because our tools and teams aren’t equipped to manage the scale of big data, we shy away from diving in at a transactional level.
There are, however, three critically important reasons you should equip your business to produce insights from the lowest levels of its data.
1) Identify immediate opportunities to make money, save money, and manage risk.
An anomaly represents a significant departure from a forecast or expectation. Systematically surfacing and reviewing these spikes or dips in your business performance can alert you of new and immediate opportunities to make money, save money, and manage risk.
There may be thousands or even millions of metrics used to manage your business performance, and a team of analysts will never have the capacity to analyze and spot changes across all of them using dashboards. What’s more, the current approach to visualizing data relies on data smoothing, which regularly masks small yet important variations and seasonal effects.
Given these challenges, automated anomaly detection has become a formal technique of machine learning, allowing businesses to augment the curiosity of their analysts with machine-led processes. This approach ensures you always know what’s happening across all your performance metrics, at the lowest levels of data.
2) Understand what’s truly driving your business results to make more effective decisions.
The first question a business leader should ask when notified of an anomaly is, Why? Why are sales up? Why is customer churn down?
Without understanding the root causes behind a change in business performance, you can’t be confident you’re making or recommending the correct decision about what to do next. To be certain you’ve arrived at an accurate list of causal business drivers, you must begin your analysis with transaction-level data, rather than summarized or aggregated data which may bury the true storyline.
3) Know what’s likely to happen next before making or recommending a decision.
Estimating the potential impact of a decision before you make it is crucial, and doing so accurately requires applying your knowledge of what’s changed and why to the lowest levels of your data in a bottom-up approach.
For example, if you’re forecasting operating margin, you might look at margin’s historic performance. That makes sense. However, if you don’t know what’s happening to margin’s drivers (like revenue) at a granular level, you may miss an increase in churn (i.e. a decrease in the customer base) after a price increase. In other words, you need to know what’s happened at a customer level to know what will happen at the company level.
Consider the last report, analysis, or dashboard you looked at. Was it produced by a comprehensive understanding of your little data? Is your team equipped with the right tools to surface insights from the lowest levels of your data?
