Spark Streaming wasn't the first streaming architecture. Over time, multiple technologies have been developed in order to address various real-time processing needs. One of the first popular stream processor technologies was Twitter Storm, and it was used in many businesses. Spark includes the streaming library, which has grown to become the most widely used technology today. … [Read more...] about Spark Streaming: Understanding StreamingContext
apache spark
Python, Java or Scala? What to Use for Your next Spark Project?
Apache Spark is a leading general data processing platform that runs programs 100 times faster in memory and 10 times faster on disk than the traditional choice for Big Data applications, Hadoop. Spark is a project from Apache that the company likes to sell as a lightning fast cluster computing platform. A dilemma amongst the developers and users of the Spark platform is … [Read more...] about Python, Java or Scala? What to Use for Your next Spark Project?
Big Data World, Frankfurt
The newest phenomenon in your big data universe As businesses increasingly operate in the cloud and harness the groundbreaking benefits of IoT, they are creating an ocean of data. Welcome to Big Data World, the one place you can immerse yourself in it. The true value of data is the ability to contextualise, understand and employ it to your advantage. Big Data World will show … [Read more...] about Big Data World, Frankfurt
Real-time stream processing: Are we doing it wrong?
Humans to Machine “ Shift of data source Data has been growing exponentially. We have more data streaming through the wire than we can keep them on disk from both value and volume perspective. These data are being created by everything we deal with on daily basis. When humans were the dominant creator of data, we naturally used to have fewer amounts of data to deal with … [Read more...] about Real-time stream processing: Are we doing it wrong?
What Everyone Should Know about Machine Learning
Over the last few months I've had the opportunity to talk to a lot of decision-makers about artificial intelligence in general and machine learning in particular. Several of these executives had been asked by their investors about their machine learning (ML) strategies and where they have already implemented ML. So how did this technical subject all of a sudden become a topic … [Read more...] about What Everyone Should Know about Machine Learning
