For data teams, bad data, broken data pipelines, stale dashboards, and 5 a.m. fire drills are par for the course, particularly as data workflows ingest more and more data from disparate sources. Drawing inspiration from software development, we call this phenomenon data downtime- but how can data teams proactively prevent bad data from striking in the first place? In this … [Read more...] about 5 Strategies For Stopping Bad Data In It’s Tracks
data quality
Data 2030 Summit MEA 2023
Data 2030 Summit MEA is an annual strategy event gathering the Data Management community in one place to discuss ways of enabling faster Data Innovation and AI deployment across the enterprise by setting up a modern Data Management strategy and platform for the new decade.' ' With 20+ international speakers presenting across two days, panel discussions and plenty of networking … [Read more...] about Data 2030 Summit MEA 2023
Assuring Data Quality: How to Build a Serverless Data Quality Gate on AWS
Data is a vital element in business decision-making. Modern technologies and algorithms allow for processing and storage of huge amounts of data, converting it into useful predictions and insights. But they also require high-quality data to ensure prediction accuracy and insight value. In today's world, the importance of data quality validation is hard to overestimate. For … [Read more...] about Assuring Data Quality: How to Build a Serverless Data Quality Gate on AWS
Data Lineage is Broken – Here Are 5 Solutions To Fix It
Data lineage isn't new, but automation has finally made it accessible and scalable-to a certain extent. In the old days (way back in the mid-2010s), lineage happened through a lot of manual work. This involved identifying data assets, tracking them to their ingestion sources, documenting those sources, mapping the path of data as it moved through various pipelines and stages of … [Read more...] about Data Lineage is Broken – Here Are 5 Solutions To Fix It
You Can’t Out-Architect Bad Data
Say it with me: bad data is inevitable. It doesn't care about how proactive you are at writing dbt tests, how perfectly your data is modeled, or how robust your architecture is. The possibility of a major data incident (Null value? Errant schema change? Failed model?) that reverberates across the company is always lurking around the corner. That's not to say things like data … [Read more...] about You Can’t Out-Architect Bad Data