Big data was still big news for much of 2017, thanks in part to the accelerated growth of the Internet of Things (IoT). The proliferation of smart devices we saw being put to practical use last year “ at both consumer and business level “ meant we were constantly being reminded of the speed at which big data was, in layman’s terms, getting bigger’.
However, another trend emerged during 2017 that runs almost counter to the above: as this site noted in a round-up of emerging trends at the end of last year, the concept of big data itself arguably ceased to be news at all. Between 2017’s constant talk of smart home products, smart customer analytics and service, smart production line hardware and smart logistics solutions, we became inured to those once-newsworthy statistics about the sheer volume of information.
Indeed, the idea of big data itself became almost mundane; most companies simply reverted to calling it data‘. It ceased to qualify as a buzz term.
Real-world outcomes
If increasingly widespread acceptance was a defining feature of 2017, we could expect 2018 to be about focused, real-world applications. In short, the impact of big data is set to become far more tangible over the coming months than ever before.
So far, to the majority public at least, big data has tended to remain a somewhat woolly concept “ a vague understanding that, due in part to the IoT, more and more of our daily interactions are being logged to colossal, ever-growing spreadsheets whose vast potential we haven’t yet figured out how to fully unlock.
2018, by contrast, is all about the real-world application of that information, and how it can make a practical difference to our everyday lives. In turn, we finally see the typical conversation around big data begin to shift from a spreadsheet-based, quantitative phenomenon to more of a qualitative real-world one.
Jobs and workforces
2017 was a huge year for the concept of machine learning. Along with impressive leaps forward in the complex technology associated with teaching computers to interpret and react to emerging datasets independently, we also saw increased public concern about how exactly this might impact on the job market in future.
However, things are looking far from bleak in that regard. While there will, of course, be some traditional, relatively basic jobs that eventually lose out to AI as it continues to infiltrate more of our routine daily interactions, we’re also likely to see a huge uptick in the number of positions becoming available because of big data.
Indeed, IBM projections indicate that some 2.7m people will be employed indirectly data-related positions by 2020, some 700,000 of which will specifically be in advanced data science and analytics roles. Outside of the data science realm itself, there’s the simple fact that nearly all major corporations are now implementing “ or at the very least exploring “ practical measures to harness the power of machine learning and big data principles.
This essentially means there will be very few office-based positions that don’t eventually require some degree of comfort with data practices; a structural challenge that businesses of all types are going to have to prepare for sooner rather than later. We’ll see far more of them starting to take practical steps towards this as the year progresses.
Streaming
Information capture, analytics and reaction times are all being sped up thanks to an increased focus on fleet-footed streaming platforms such as Amazon Kinesis. In the past, analysis of collected data would effectively be a static snapshot reflecting a specific point in time “ which, for obvious reasons, quickly becomes obsolete.
Now though, streaming platforms such as Kinesis and Apache Kafka are starting to better enable complex datasets for metrics such as server activity, website clicks and geolocation to be visualised and processed in real time, even as they’re being generated. This creates a situation whereby on-the-fly reaction becomes much faster to judge and implement, such as tweaking logistics to deal instantly with unforeseen hiccups in the delivery chain, serving customers with recommendations based on location data, or adjusting brand positioning in response to shifting social media attitudes during emergent scenarios.
Moreover, as the process of leveraging real-time data into on-the-fly adjustment gets quicker, it becomes far easier for companies to apply literal cost-impact analyses to those adjustments. Streaming should effectively move us a significant step closer to proper monetisation of big data in 2018.
Humanisation: big data vs small data
Ironically, one of the key trends we fully expect to see in big data this year involves companies trying to make it smaller; not physically smaller, but with an increased focus on the personal and the individual. This process is often referred to as data humanism’, and it frequently relies heavily on visualisation to provide greater insight into the near-infinite range of complexities and nuances that collectively generate those larger numbers.
Again, this is a key factor in the ongoing shift towards qualitative, rather than quantitative, approaches to big data analytics. Many leading retailers and service providers are already finding ways to leverage masses of gathered data in serving up a more bespoke, tailored approach to individuals and their experiences: 2018 is going to be all about the customer journey, and we’ll notice its impact more and more over the coming months across a wide variety of commercial end-user scenarios.
