The Foundational Blog

In this blog post we focus on some of the common pitfalls when working with dbt at scale, and in particular on the limitations of dbt in detecting SQL errors, and how to effectively navigate these challenges.

We need to have an accurate data lineage graph every time new code is committed, while also understanding the lineage impact the commit has on the existing system. In this post we discuss some of the challenges in doing source-code based data lineage at scale, focusing on SQL.