Like bean dip and ogres, layers are the building blocks of the modern data stack. Its powerful selection of tooling components combine to create a single synchronized and extensible data platform with each layer serving a unique function of the data pipeline. Unlike ogres, however, the cloud data platform isn't a fairy tale. New tooling and integrations are created almost daily … [Read more...] about How to Build a 5-Layer Data Stack
big data engineer
All I Want To Know Is What’s Different – But Also Why and Can You Fix It ASAP?
I link to Benn Stancil in my posts more than any other data thought leader. I might not always agree with his answers, but I almost always agree with his questions. True to form, last week he tackled one of the most important questions data leaders need to ask which is, "How do we empower data consumers to assess the credibility of MDS-generated data products?" The question … [Read more...] about All I Want To Know Is What’s Different – But Also Why and Can You Fix It ASAP?
Data Freshness Explained: Making Data Consumers Wildly Happy
What is data freshness and why is it important? Data freshness, sometimes referred to as data timeliness, is the frequency in which data is updated for consumption. It is an important data quality dimension and a pillar of data observability because recently refreshed data is more accurate, and thus more valuable. Since it is impractical and expensive to have all data refreshed … [Read more...] about Data Freshness Explained: Making Data Consumers Wildly Happy
The Next Big Crisis for Data Teams
Over the past decade, data teams have been simultaneously underwater and riding a wave. We've been building modern data stacks, migrating to Snowflake like our lives depended on it, investing in headless BI, and growing our teams faster than you can say reverse ETL. Yet, much of the time we didn't know whether or not these tools are actually bringing value to the … [Read more...] about The Next Big Crisis for Data Teams
Ready or Not. The Post Modern Data Stack Is Coming.
If you don't like change, data engineering is not for you. Little in this space has escaped reinvention. The most prominent, recent examples are Snowflake and Databricks disrupting the concept of the database and ushering in the modern data stack era. As part of this movement, Fivetran and dbt fundamentally altered the data pipeline from ETL to ELT. Hightouch interrupted SaaS … [Read more...] about Ready or Not. The Post Modern Data Stack Is Coming.
