Spark Streaming wasn't the first streaming architecture. Over time, multiple technologies have been developed in order to address various real-time processing needs. One of the first popular stream processor technologies was Twitter Storm, and it was used in many businesses. Spark includes the streaming library, which has grown to become the most widely used technology today. … [Read more...] about Spark Streaming: Understanding StreamingContext
Big Data is not just about volumes of data; it is the ability to both collect information and extract meaning from it. The businesses that are prospering in today's economy are those able to take advantage of Big Data. In the next ten years, it will be the ability to use Big Data that will make or break a business. According to contributing writer to Forbes, Daniel Newman, "it … [Read more...] about Big Data, not Oil, is King in Today’s Economy
Before diving deep into how Apache Spark works, lets understand the jargon of Apache Spark Job: A piece of code which reads some input from HDFS or local, performs some computation on the data and writes some output data. Stages: Jobs are divided into stages. Stages are classified as a Map or reduce stages (Its easier to understand if you have worked on Hadoop and want to … [Read more...] about Apache Spark – A Basic Understanding
Disclaimer: The results are valid only in the case when network attached storage is used in the computing cluster. The amount of data is growing significantly over the past few years. It is not feasible for only one machine to process large amounts of data. Therefore, the need of distributed data processing frameworks is growing. It all started back in 2011 when the first … [Read more...] about Apache Flink: The Next Distributed Data Processing Revolution?
One thing that could have a deep impact on business is deep learning. Deep learning can be thought of as a subfield of machine learning. In specific, this form of machine learning was influenced by the study of the human brain. The algorithms involved are designed to mimic how the human brain operates to allow a machine to learn in the same way. This is done through a system … [Read more...] about What Effect Will Deep Learning Have on Business?