Foundational gives us confidence when making changes. It allows us to get end-to-end lineage, from the application through Snowflake and dbt, and all the way to LookML and Looker.
Company: Lemonade (NYSE: LMND)
Industry: Insurance
Lemonade is disrupting the insurance industry by providing a charitable business model that pays out claims quickly and offers hassle-free sign-up, claims filing and customer service through its mobile app. The company offers insurance for Homeowners, Renters, Car, and Pet owners.
This case study demonstrates how Foundational — a global leader in source-code lineage, data quality and governance — provided its products and services to introduce lineage and preventative data quality to Lemonade’s BI and Analytics groups, and improve the developer workflows around data and business intelligence.
Lemonade uses Snowflake for its data platform, together with Looker as the visualization tool. It heavily uses LookML together with dbt for transformations, which together introduce quite a bit of complexity as the company has grown to a massive scale. Lemonade’s scale is impressive – It has thousands of dashboards and tables across multiple applications and lines of business. The data platform is used for mission-critical use cases such as financial reporting, fraud detection, auditing, and more.
As a public company, Lemonade has grown rapidly while continuously introducing new business offerings to the market, powered by innovation, product development and M&A. These and others are contributing factors to the Lemonade data platform being complex, containing silos and unknown dependencies and even some legacy workloads.
Lemonade faced a big challenge: People in the data organization did not have a good understanding of end-to-end lineage and downstream impact while making code and product changes. For example, the team did not have good visibility to the dependencies between assets in Snowflake and views in Looker and LookML. All of these have resulted in the team moving slower, having data quality issues, and generally needing to spend too much time debugging and addressing problems.
The team looked into building out a custom solution but that would have not given the same results and would have required too much ongoing maintenance. Lemonade’s data team identified Foundational as the selected partner to provide downstream impact analysis, introduce the concept of preventative data quality to data engineering builds, and provide end-to-end lineage for its data platform.
Foundational stands out in the complex world of data quality and lineage solution with our unique approach to preventative data quality, and enabling CI and downstream impact for every part of the data development lifecycle.
The technology built by Foundational allows for easy integration without any code changes needed, ensuring a quick onboarding process that covers every part of the stack.
Foundational’s source-code based lineage is unique, allowing it to analyze pending code changes as opposed to existing lineage and data quality solutions that are reactive, and limited to the data warehouse.
The Foundational team are world-class experts in reverse engineering and source code analysis, who put customer satisfaction and user value as the top priority.
Our expertise is not just in handling the tools and technology but also in understanding real-world deployments and constraints, being able to adapt to various challenges when analyzing complex data environments.
With Foundational, Lemonade found a partner who allowed it to scale even faster with minimal concerns around legacy parts of the stack, data quality, and shipping bad code.
Foundational partnered with Lemonade’s BI and Analytics teams to execute a better lineage and developer experience that automates data lineage across Snowflake and Looker, as well as checks and validates every new piece of code introduced to Snowflake, dbt and Looker. Our holistic solution incorporated three core components:
Our integrations with dbt, Snowflake and Looker, including LookML, have allowed Foundational to automate data lineage across all of these tools. While traditional lineage tools rely on query logs, Foundational’s source-code based approach allows for real-time lineage, better accuracy, and improved visibility to non-warehouse flows.
Foundational’s integration with GitHub also allows for the Downstream Impact Analysis to get communicated through GitHub comments against every pull request, providing visibility to analytics engineers to the cross-platform impact of their code change, before deploying it. This visibility provides confidence and replaces the need for manual checks, ultimately costing time and money to Lemonade.
Foundational enables Lemonade to identify data quality issues pre-deploy. Prior to Foundational, Lemonade relied on open-source validation checks to validate some aspects of the code, not really knowing the full extent of a given code change across all platforms.
Foundational’s Data Quality checks introduce an additional layer of data quality that checks the code before its deployment, to identify potential data issues and prevent incidents.
With Foundational, Lemonade was able to get full visibility to every dependency between Snowflake and Looker, and remove any knowledge barriers holding back the various teams who are constantly creating and updating business logic to support Lemonade’s business priorities.
The third part of our solution allows Lemonade to create policies against unknown changes that impact important data assets, such as Looker views. This functionality allowed the Lemonade team to to minimize the chances of upstream code changes which would get merged without knowing the impact on important downstream data assets.
Foundational uses a connector-based setup and does not process or extract any data or user records. This means that the only access we get is to code and configurations, which provides for better data security and access controls.
Foundational natively integrates with git, which in Lemonade’s tech stack is provided by GitHub, to power the developer experience directly in the interface developers use, regardless of their department in the organization.
The deployment was an overwhelming success, marked by clear improvements in KPIs and positive outcomes:
Most importantly, the team at Lemonade is satisfied. When asked about the integration, Qun Wei, Lemonade’s VP of Data Analytics, said, “Foundational gives us confidence when making changes. It allows us to get end-to-end lineage, from the application through Snowflake and dbt, and all the way to LookML and Looker. It’s now also easy for us to migrate the older pipelines as we rebuild the data warehouse into a single source of truth.”
The success of this integration in helping data quality and cross-platform lineage against a complex, highly dynamic environment, is a strong testament to Foundational’s approach of looking directly at source-code as a new paradigm in data management and improving data quality.
As your organization considers different solutions to improving data quality, let Foundational become your partner to shift-left governance, preventative data quality, and automated data contracts. Our approach allows for minimal setup time, extended coverage, and improved developer experience when working with data.
