Big data is only getting bigger. Last year weve collected more information than we did in the whole of human history before that. Even more frightening, that data is doubling every year. Thats a lot of data!
That means two things:
1. Ever more can be understood on the basis of big data.
2. The longer you wait with joining the big data the game, the harder it will be for everybody else will be further ahead of you.
And when I say everybody I do really mean everybody. From logistics to healthcare, from finance to logistics, from multinationals to small businesses, big data is a part of their strategy and informing the decision that theyre making.
Therefore, isnt it about time that you joined in as well?
Of course, the image of my audience in my mind grumbles, but how do I do that?
Get the right team together
The first thing to realize is that as big data is still quite new, there isnt yet anything like a big data person. Whats more, it isnt just some IT program. Instead, its a business strategy. For that reason, you need to make sure that your team has all the necessary skills to actually make the best of the big data initiative youre putting together.
That means that you need people who can interpret big data as well as write the algorithms to draw in the data you need. Youll also need business strategists to inform what data youll need as well as to execute based on the ideas. Youll need people who understand finance to measure feasibility, as well as people who understand development, logistics, and production.
Basically, if its an important part of the company, then youll need somebody who understands it in your big data team.
Ask the right questions
Next, youll need to decide on a direction. The best way to do that is to find three problems youd like solved, then turn those into three questions. Create shared docs on Google docs or elsewhere in the cloud that everybody in the team can edit. In that way, the whole team can get involved in deciding if the question is too narrow, too broad or if youve hit the goldilocks.
Remember, the first project is a test run, so dont shoot for the moon. Instead, concentrate on manageable questions that can be answered in a realistic frame of time. In that way, the team can find their feet and learn from their mistakes.
Analyze what data youve got
Since data is the resource that will be analyzed, its important that you know where you stand in terms of data. What is easily accessible? Where are the gaps in the data you have? Where has data been stored differently? And so on.
This will not just give you a good insight into what is possible, but might also reveal opportunities that you might not have been aware of.
Find more data
Of course, to answer your question your internal data set probably isnt enough. For that reason, youll need to go outside and see what additional data you need. There are many places you can look. Social media platforms will sell you data, while often you can get it from government servers for free. Data Gov, for example, has over 100,000 data sets. Yup, thats a lot! So check out all the lists available.
Collect what you need. Realize that in these kinds of situation youre only ever as good as your data. And, as they say, garbage in garbage out. So dont skimp on this part if you want your project to actually have a reasonable chance of success.
Also, dont go overboard. Data can become unmanageable if you collect too much of it. So, start with a few data sets and only expand if you find that that really isnt enough.
Frame how you want your output
Youve got your input, now its time to define your output. How will you answer the question? When will you be satisfied? These are important questions that will help guide your teams questions and inform their processes.
If you dont ask these questions and give well-considered answers, there is a good chance your team will flounder and not be able to find the right direction. So take the time necessary to actually answer this question.
Get expert advice
Since this is your first time out, make sure that youve got some people that can answer the difficult questions that crop up. What youre looking for is people who have a lot of experience with big data and can guide you and your team in the right direction.
Sometimes youll have to pay for this kind of guidance, but ultimately that will be a lot cheaper than having your whole team not being able to continue because theyre missing some key ingredient or dont understand how to deal with a problem.
The best strategy is to get them involved early and let them look over what youve decided to do and how youre going to do it. In this way problems that you werent even aware of can be dealt with before they cost resources and time.
Last words
And thats it. Your team is ready to go and your first question is ready to be answered. Now its just a matter of letting them get on with it. The best part is that just like any other process, the more often your team goes back and tries to answer questions, the more experienced theyll become.
In this way, your questions can become more intricate and the answers more valuable over time. Thats another good reason to start early so that when the really important questions arise, your team is already ready to deal with them.
