“Big Data is one of those words that seems to be everywhere at the moment, used by everyone from Data Scientists to Branding Analysts. Yet, until now, it has remained merely a buzzword for many companies.
It seems, though, that things are changing in Australia. Recent research has found that Australian companies are spending more than ever before on Big Data systems and research, and that this stream of investment is projected to continue.
This is a welcome development, of course. Implementing Big Data systems is now easier and cheaper than ever before, and can bring great benefits to any enterprise that takes it seriously. Even small companies are embracing its power, and it is making itself felt in many industries that were previously sceptical about it.
The Growth of Big Data
New research by Telsyte suggests that Big Data is going from strength to strength in Australia. Conducted in the early part of 2017, as part of this research Telsyte talked to a huge range of companies in order to ascertain their attitude toward Big Data, and whether they would be investing more in this area in the future.
The results were striking. 83% of the Australian CIOs that Telsyte spoke to are planning to invest more in Big Data in the coming year. The range of applications being considered by these companies are very broad. Most planned to bring some form of Big Data analysis into their financial modelling, customer interaction, security and fraud detection, retail and e-commerce systems. Another welcome development “ and especially so for professionals working in this area “ is that IoT and machine-to-machine infrastructure projects are also being considered.
In addition, around a third of CIOs are looking to integrate Big Data systems into their sales and marketing applications, making it one of the top three spending lines. To date, however, only 15% of marketing departments in Australia are actually using Big Data.
This points to a broader problem with surveys like this. Big Data means many things to different people and companies, and what some firms call Big Data is merely a larger spreadsheet. At the executive level, understanding of true Big Data systems remains pretty slim, and so often a commitment to spend more in this area is motivated more by companies wanting to be seen to be progressive, rather than an actual desire to develop.
Low Maturity
And in fact, although Big Data systems are being implemented by more and more companies, in the majority of cases these systems remain under-utilized. As part of their research, Telsyte also looked at the maturity of the Big Data systems already in use in Australia, and ranked them according to its maturity model.
They found that 63% of enterprises are still at a low maturity level. These companies have made the decision to implement Big Data systems, and in some cases these systems are relatively sophisticated. However, they are not yet integrated into decision-making processes: 7% of companies, for instance, merely use a static system to look back at prior business trends, rather than using their expensive Big Data system to predict and plan for the future.
On the other hand, the number of companies that have a mature system, according to Telsyte, has increased to 37%, up from the 29% of 2014. How much this reflects actual spend and development, however, is difficult to measure. A lot of IT providers now incorporate Big Data elements into their products, such as Digital Pacific, who’s web hosting services come with Big Data analytics as standard.
The survey itself reflects this point, finding that Big Data analytics is now one of the top five enterprise applications that is managed by third-party providers. Telsyte suggests that this statistic represents a lack of in-house capability to deal with large data sets, and it is hard to argue with their reasoning. In this respect, the increased maturity of Big Data systems might merely reflect increased uptake of off-the-shelf systems, rather than an active development of such systems by firms themselves.
But Is It All Good News?
This point gets to the heart of the current problems with using Big Data. A lot of firms, eager to jump on the bandwagon, seem to end up with generic systems that are not that well suited to their needs.
In addition, and as Datafloq has long pointed out, implementing a Big Data system is more about how you use data, rather than the expensive hardware and software used to collect it. In fact, in a lot of cases employees already have access to vast amounts of very useful data, but this is not available to those making strategic decisions about the future of companies.
The best way to approach a new system, therefore, is to start with a specific problem and find out if you are already collecting enough data to solve it. In a lot of cases, all that is required to implement an effective Big Data system is to aggregate the data you are already collecting in a central place, and then do some basic analysis on it.
It doesn’t stop there, however: you then have to use this information in a smart way, and also be aware of the limitations of any such system. Those companies that have long had an excellent system for collecting and using Big Data sometimes forget that it is not a solution for every problem they have, and some people have even warned about the false sense of security that Big Data brings with it.
Overall, then, while the increased numbers of Australian firms using Big Data is to be welcomed, this comes with the usual warnings about how and why this data is being collected, and how it is being used. Big Data is certainly a very powerful tool for any kind of business, but to work properly it needs to be dynamic, and seamlessly integrated into decision-making processes.
