Will Big Data Mark The End Of The Market Research Industry?

Big data is disrupting the world, society and all industries. One industry that will definitely notice these waves of change is the market research industry. The new big data startups, that are founded everywhere around the world, can be viewed as the new, but improved, market research agencies. Consequently, the market research industry as we know it today will seize to exists.

Customers interacting with organisations provide valuable feedback for companies. With big data this feedback, whether provided via the website, via the call centre or by simply using the product, can be turned into market information for product improvements. With big data it is possible to turn every conversation recorded by a call centre into text, which could then be data mined for insights. This would provide very useful content about what and how consumers feel about the brand and the products.

Even more, consumer panels, panels of business managers or focus groups will not be necessary anymore in the world of big data. In an inter-connected world, where products have sensors and are connected to the internet, companies will know in real-time how people use their product, when they use it, for how long they use it and when things go wrong. Whenever a product needs to be improved or a new product needs to be developed, organisations can simply look at the real-time sensor data pouring into the organisation and understand the needs and wishes of their customers. If they want to understand how people think of their products, the company or what the sentiment of the brand is, they can simply connect to a big data startup and real-time data starts appearing on their screen, analysed and visualized, to be understood by everyone in the organisation. A market research organisation that requires a few weeks to create and perform a questionnaire and another few weeks to analyse the results, is likely to be by-passed in this fast world.

In a blog post by Brigid Kilcoin it is noted though that market research agencies can actually thrive due to big data because market research organisations focus on the why. She writes that it is only in understanding the context of why something is happening, that an enterprise can drive meaningful change within its organization. According to Kiloin, market research organisations are there to provide an answer to this why question. In a world of big data, however, understanding of the context is done automatically by the algorithms. Members of an organisation can use the analyses and visualizations made by the big data tools to under the context of why something is happening.

In a big data society, researchers are consequently not needed anymore to tell a compelling story using visualizations. There are great tools available on the market already that can make these visualizations automatically and deliver them in real-time to the decision-maker.

Gigi DeVault, from about.com, notes that all theories of human behavior can be tossed out since data analysts no longer need to know why people do what they do. Human behavior can be tracked and measured with “‘unprecedented fidelity.” Market research organisations will therefore have to adapt to the new rules of the game or they will be by-passed by big data startups who can do the work for market research agencies at a fraction of the costs and in real-time.

It will require market researchers to understand the power of Big Data and to become comfortable with the tools and techniques of Big Data. If they want to survive, they should start hiring big data scientists to be able to deliver the same kind of services as big data startups around the world already offer.

So yes, big data will mark the end of the market research industry as we know it. Unless, of course, they are able to adapt and pivot on time. Otherwise, the big data scientist will replace the market researcher and this will happen faster than one might expect.