How to build a Successful Big Data Analytics Proof-of-Concept

This article is sponsored by CloudMoyo – Partner of choice for solutions at the intersection of Cloud & Analytics.

For all kinds of organizations, whether large multi-national enterprises or small businesses, developing a big data strategy is a difficult and time-consuming exercise. In fact, big data projects can take up to 18 months to finish. While a few within an organization may be very well aware of what Big Data is and what the possibilities of Big Data are, not everyone else, including the decision-makers, are aware of this. Developing a proof of concept (PoC) is a right approach to begin with and develop a business case. This can help organizations to answer questions like where to start, departments to be involved, functional areas to be addressed, and what will be the return on the required investment. All of these aspects should be involved in your big data business case.

Buying Business Intelligence (BI) and Analytics solutions has modified dramatically within the past few years with the advent of emerging technologies and the ever increasing sources of data. Historically, vendors didn’t supply a Proof-Of-Concept (POC) stage throughout the buying process and if a vendor did provide one, it’d typically take months to line up and costed a fortune to launch.

Once the context is set, it is important to create the right objectives of the Proof of Concept. A Proof of Concept is not just about financial objectives, but also about the learning experience for the organisation. Big Data requires a different way of working and a different culture. It not only involves new hardware and software to use, but access to real-time insights in whats happening within an organisation. Over-confidence can backfire when the Proof of Concept does not achieve its set objectives and you cannot move ahead with more Big Data projects. For the buyer, the misalignment of interests often results in a disappointing PoC and a high cost. To avoid this, here are tips that will put you on the right track of conducting a successful Big Data Proof of Concept–

• Use your own data– A PoC built on sample data does nothing. Lot of vendors try to reduce the number of data sources or the volume of data for a PoC. However, all organizations have their unique demands and challenges. For a PoC to succeed, using own real data is the best way to prepare a business case and chalk the plan ahead.
• Involve Big Data distribution vendor at the POC conceptualization stage Cloudera, DataStax, Hortonworks, IBM, MapR, Google – whatever distribution of Hadoop / Big Data you are choosing for the POC, it is a must to involve these companies from conceptualization stage
• Pretty aint everything– Visualizing data is importatnt but most PoCs tend to focus on a few static but beautifully built dashboards. More often than not, these are sample dashboards which can be cooked easily by a vendor using multitude of visualization software available. The real challenge is to have self-service, easily configurable customised dashboards on your own data so that one does not spend a lot of time and money later on this aspect.
• Define governance Big Data projects can turn out to be one of the most multi-dimensional endeavours in the organization. There will be a constant need for business users to comment on whether additional investment in Big Data is resulting in multi fold increase in decision making capability. The list can go on and on. All these decisions need to be orchestrated from an organizational perspective by a committee of senior executives. Having a Big Data governance council is a must.
• Proving value- For your POC to succeed in six weeks, you need to translate expectations of success into clear metrics for success by listing qualitative and quantitative measures
• Scalability– The PoC should be able to address future requirements with minimal effort and not just some old reporting need. BI requirements tend to be highly dynamic because businesses change all the time and business users are continually refining and adjusting their requirements. Todays reporting needs will look very different in a year from now, and todays analysis will likely be relevant for only a short period of time before becoming obsolete.
• Involve corporate IT– In order to reduce lead time and dependencies on corporate IT, most business users dream of an analytics solution that can be a self-service tool without the need for intervention from IT. However, it is still highly recommended that you consult your organizations IT professionals regarding topics with which they are more familiar: scalability, integration cycles and so forth. They can help to achieve a better architecture with their inherent know-how of companys IT landscape and pain areas.

What Components Should You Consider in Your Big Data PoC?

• Big Data Storage and Processing
• Real-Time Ingestion
• Data Virtualization and Federation
• BI, Reporting and Visualization
• Analytics
• ETL / ELT Data Integration
• Data Discovery and Exploration
• Data Governance

CloudMoyo has delivered successful Business Intelligence & Analytics projects for its clients across multiple industries such as healthcare, transportation, pharma, retail. A lot of this success can be attributed to a thorough assessment of client landscape followed by a proof of concept on real live client data. Most of these clients were able to pursue their big data projects after a successful PoC. With its expertise in deploying cloud based analytical solutions, CloudMoyo is the right partner for you to engage for your Big Data proof of concept. Book your free Big Data assessment now!