Dangers and Missed Opportunities: Using Historical Data in the Age of Real Time Decision Making

For many companies going down the big data path, collecting data for its own sake has been a costly exercise with varying degrees of ROI. 

Historic data is great for understanding past events, but as companies mature, their big data efforts move to become more central to business operations: from descriptive, diagnostic analytics to predictive, prescriptive courses of action. In this environment, understanding how real-time external data sources benefit an organization is vital. 

Historic vs. Real Time Data

Traditionally, companies engaged management consultants or in-house data scientists to pour over copious amounts of historical, internal data to identify trends and develop models and algorithms to predict future trends and improve the companys operations.  

The problem with historic based models is that without current real-time data, companies cannot effectively prescribe a course of action. To remain competitive in today’s business climate, companies need current conditions, which are ever changing and evolving. Historic based data models need real-time triggers.

In real world terms, past traffic data can generally tell you what the fastest average route through a city current traffic conditions require the knowledge of real-time factors including accidents, road blocks, weather conditions and special events to effectively guide vehicles through a city. 

By knowing the key real-time factors that might impact a model, data scientists can now develop algorithms which take into account key data points to arrive at a more accurate prediction of future conditions and create actionable items based upon them. The combination of real-time events with historic patterns allows predictive and prescriptive analytics to emerge. Such evolutionary analytics allows data scientists and managers to solve issues and prescribe solutions in real-time.

Data analytics firm Vitria (https://www.vitria.com/) conducted a study of 235 US based firms and found that 25% are already embarking on real-time predictive modelling; while another 47% are actively exploring the opportunity, with the remaining 28% still sitting undecided how to proceed. The findings are clear: while there is sizable interest and utilization of real-time data analytics, there is still a lot of room for growth and maturity in the field.

Internal vs. External Real-Time Data

To further enhance real-time models, organizations are now combining both internal and external real-time data. Historically, many models only used the company’s own internal data resources, but now, more and more organization are finding the only way to more accurately predict the future is to understand the external factors that impact business processes.  For example:

Footfall Data on Impact of Sales

It is no surprise that historic data on where businesses see the most customers and revenue can be useful.  However, real-time footfall patterns (the movement of people throughout a given geography) may vary wildly due to a number of factors such as weather, special events and traffic conditions. Not taking these external factors into account means that companies, large and small, are missing the opportunity to maximize revenue and operational efficiency. As an example, by understanding external, real-time factors effecting population movements, mobile food trucks could deploy more efficiently to various locations at any given time. 

Hospital Resource Allocation

Previously, hospital human resource allocation had been based upon numerous factors including current patient load, predicted number of new patients, and resource availability. However, hospitals are now combining internal information with external real-time sources including: weather, disease trends and sister-hospital patient load in order to make life-saving decisions around human resource allocation and staffing levels. 

Pricing Data on Impact of Sourcing

Historic data on prices of produce do not provide accurate predictions of current prices due to a variety of factors such as: weather, water supply, logistics, soil conditions and disease.  In contrast, real-time pricing data could provide a much stronger base for buyers to negotiate prices with produce sellers.  

Weather Data on Impact of Advertising

As referenced in the Stella Artois case study from WeatherUnlocked.com (see below), weather has a major impact on consumption patterns across a wide range of industries including: food & drink, travel, apparel, healthcare, home & garden, energy and automotive.  By leveraging real-time weather data, marketers can develop more timely, relevant, impactful communication campaigns and optimize media spend and geographic touch points.  

Stella Artois Case Study

By mapping historical weather data against sales metrics for its Cidre product, Stella Artois discovered that a two degree temperature rise above the monthly norm triggered a rise in sales.  Digital out-of-home billboards were identified as the optimum channel due to their proximity to stores.  Using real-time weather data, Stella Artois ran a campaign on a cost-per-minute basis, activating only when conditions were right to induce purchase intent.

Results of the campaign were astounding:  

• Stella Cidre achieved a 66% increase in YOY sales during the period of the weather-responsive campaign(source: Nielsen Sales data 2013)
• Cost efficiencies were up by 50% because ads were only shown during optimum conditions, thus there were no wasted impressions
• The campaign created a lot of buzz and online engagement, generating over $30,000 in earned media value

Streaming Forward 

Given the trillions of bytes of streaming data that are now available, focus is key for informed organizations to be successful. Deciding which external data sources are most valuable and how best to exploit them will be a key determinant for future success.  Creating models based on historic data is only the first step in improving a companys operations. The true insights come by combining both internal and external real time data sources in creating actionable decision-making models companies. In this way, companies across industries can receive a better ROI on their big data efforts.

Image: EMC