Companies need data scientists to provide them insight and direction of structured data
We live in a world practically drowning in data. As Co-founder and Co-director of the MIT Initiative on the Digital Economy, Andrew MacAfee once said, The world is one big data problem.
Indeed, people who have been in business for at least a decade, can see a distinct change in their careers Before Big Data (BBD) and After Big Data (ABD). In early times before the advent of the Internet, data in companies came from mostly internal sources, and was inevitably small and simply structured. In addition, analysts were focused more on collecting and preparing data, than actually analyzing and providing insight. Data analysis was significantly limited during this time, up until around 2009.
As Craig James Mundie, Senior Advisor to the CEO at Microsoft, said, data was gradually becoming the new raw material of business. It was around 2010, that the world, in general, began to notice the coming to being of big data. The term big data was forged by technology expert, Roger Magoulas back in 2005 , to describe a variety of massive, complex data sets almost impossible to manage and process with traditional data management tools.
The era of big data is distinctly different from the preceding era. Data is now most often received from the outside, and is either voluminous or unstructured. This means the data has to be processed fast and stored. Therefore, the need arose for quantitative analysts who are called data scientists, and who, getting into uncharted territory, needed data science training.
In contemporary businesses around the world, data analytics handled by data scientists is the driving force of organizational strategies. A senior data analyst, Sam Underwood, said, Data gathering is just step one of a data-driven strategy; the real work comes in sorting through the data to decide what to include and what to disregard or de-prioritize. Data can, for instance, be used to predict the newly discovered best treatment for a chronic sickness or to identify and counter national security threats.
With enormous amounts of data available to organizations through technological advance, there are two major methods that companies use to store their available data “ the data warehouse or the data lake. Superficially, the end result of both methods is the same. They store data until it is needed for processing and analysis. However, deeper study shows that a data warehouse is an archive of structured, filtered data, already processed for a specific purpose. A data lake, on the other hand, is a massive pool of raw data, with no purpose yet defined. The concept of a data lake was created by expert data analyst, James Dixon, in 2010. He described a data lake as a place where contents (of the data lake) stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples.
Companies struggling to make sense of excessive quantities of diverse data, understand they need smart data lake, which understands what data is there and how it can be used. This is where data scientists come in, to provide actionable insights that can have profound impact. Some companies such as in the field of education, find that data lakes offer more flexible solutions for streamlining billing and for improving fundraising at educational institutions. Healthcare companies also find greater affinity with the combination of structured and raw data. However, finance companies find data warehouses a better storage model by being structured.
According to the Belong Talent Supply Index (TSI), which matches talent availability with talent demand, there has been a 400% rise in demand for data science professionals across varied industry sectors in India.
Andrew Flowers, an economist at Indeed, based in Austin, Texas, said, The job of a data scientist has only grown sexier. More employers than ever are looking to hire data scientists.”
As Huffington Post stated, Even high schoolers are clamoring to get into the data science field ” they realize jobs in the data sector are expected to grow more than sevenfold, totaling nearly 3 million positions by 2020.
The demand for data scientists is definitely high and rising further, for they enable decision-making, which is the ultimate goal of using data. However, the success of data scientists in enabling companies, depends to a large extent on communicating on the same wave-length with entrepreneurs. A strong bond between business leaders and data scientists is essential to foster trust and confidence so vital to reach their common goals of taking the organization to the next level. Data scientists may be confronted with enormous swaths of data, but unable to effectively relate to their business leaders to explain how to use that data to become more profitable. It is like finding an antique treasure chest without the key.
As the inventor of the World Wide Web, Tim Berners-Lee, said, Data is a precious thing and will last longer than the systems themselves.
