Why Creativity is Crucial in Data Science

Data science might not be seen as the most creative of pursuits.

You add a load of data into a repository, and you crunch it the other end to draw your conclusions. Data in, data out, where is the scope for creativity there? It is not like you are working with a blank canvas.

For me, the definition of creativity is when you are able to make something out of nothing. This requires an incredible amount of imagination, and seeing past the obvious headline statistics to reach a deeper conclusion is the hallmark of a great Big Data professional.

Many of the most successful people come across as innovative thinkers when they have interviews with us. They have no choice, moulding the data in unique and unexpected ways is their job. Just as Einstein found inspiration in his violin playing, many leading data scientists find that when their creative juices are flowing, they often find the most elegant solutions to the challenges that they face. These Data Creatives are some of hardest to find candidates mainly due to the subjectivity involved. (See also a previous blog for more on Data Creatives)  

It is actually one of my favourite interview questions:

What is the most creative thing that you have done in your career?

With endless ways of interpreting vast amounts of data, the role of human creativity cannot be underestimated. As increasingly more people see Artificial Intelligence as the answer to all our data needs, I think that we will come up against a road block when humans will always have that crucial upper hand. Artificial intelligence might be able to do a good job of interpreting the data (in a fraction of the time), but it will never do an outstanding job. For that, creativity is the missing link.

Looking at the employee cultures of many of the leading global players, it is clear that harnessing their employees creativity is the key to getting them to collaborate more closely and have more empathy for each other. Data science does not exist in a vacuum, and it needs to relate to other areas in the business for it to have the maximum impact. Creatively packaging the numbers in a way that the rest of the business can understand is key to allow them to see the whole picture and therefore get behind finding the right solution.

However, there is one big but in this matter. To be creative with the data requires a decision to dive beneath the surface, and the risk that this entails. Accepting the data at face value will often be good enough, you will keep your job, and you wont ruffle any feathers with whacky conclusions. When you are creative to suggest something that is not obvious to the untrained eye, you are sticking your neck out, and this requires courage. However, you only have to do it successfully a couple of times for your wider team to give you the latitude to do so in the future. If the employer culture is resistant to such deviant thinking, it will be that little bit harder, but it is still possible.

When you approach data science in a creative way, the results are often astounding.