Data quality issues have been a long-standing challenge for data-driven organizations. Even with significant investments, the trustworthiness of data in most organizations is questionable at best. Gartner reports that companies lose an average of $14 million per year due to poor data quality. Data Observability has been all the rage in data management circles for a few years … [Read more...] about Data Integrity: the Last Mile Problem of Data Observability
big data quality
What’s Next for Data Engineering in 2023? 13 Predictions
What's next for the future of data engineering? Each year, we chat with one of our industry's pioneering leaders about their predictions for the modern data stack - and share a few of our own. A few weeks ago, I had the opportunity to chat with famed venture capitalist, prolific blogger, and friend Tomasz Tunguz about his top 9 data engineering predictions for 2023. It looked … [Read more...] about What’s Next for Data Engineering in 2023? 13 Predictions
5 Strategies For Stopping Bad Data In It’s Tracks
For data teams, bad data, broken data pipelines, stale dashboards, and 5 a.m. fire drills are par for the course, particularly as data workflows ingest more and more data from disparate sources. Drawing inspiration from software development, we call this phenomenon data downtime- but how can data teams proactively prevent bad data from striking in the first place? In this … [Read more...] about 5 Strategies For Stopping Bad Data In It’s Tracks
Data Swamps and The Tragedy of The Commons
A silent alarm rings in my head whenever I hear someone utter the phrase, "data is everyone's responsibility." You can just as easily translate this to mean that "data is no one's responsibility," too. This, readers, is what I call the "data tragedy of the commons." Here's why. The term tragedy of the commons comes from economic science and refers to situations where a common … [Read more...] about Data Swamps and The Tragedy of The Commons
Data Quality: the Silent Assassin of the Modern Data Stack
If data is the new oil, then high-quality data is the new black gold. Just like with actual oil, if you don't have good data quality, you're not going to get very far. In fact, you might not even make it out of the starting gate. So, what can you do to make sure your data is up to par? Data lakes, Data pipelines, and Data Warehouses have become core to the modern enterprise. … [Read more...] about Data Quality: the Silent Assassin of the Modern Data Stack
