Foundational helps us improve the developer experience, ship code at a faster pace; Things that we previously worried about are no longer a concern.
Company: Ramp
Industry: Financial Services
Ramp is a fast-growing financial technologies company who is disrupting corporate charge cards, spend and expense management, and bill-payment software space. The company is processing more than $10B in annual payments.
This case study explores how Foundational — a global leader in source-code lineage, data quality and governance — provided its products and services to introduce preventative data quality to Ramp’s Analytics Engineering group and improve the developer experience around data and analytics.
Ramp uses Snowflake for its data platform, together with Looker as the business intelligence tool. It also uses dbt Core for modeling and transformation, across a 1000+ model system that powers thousands of tables and hundreds of Looker dashboards. The data platform is mission-critical and Snowflake data is also used in customer-facing workflows and visualizations.
Ramp has a strong DNA of moving extremely fast. Its Analytics Engineering team has 40+ engineers who push 100+ pull requests in dbt and LookML every week, accounting to hundreds of code changes every month.
With the team and data stack constantly growing, Ramp faced a meaningful challenge: How can it continue to scale fast while making sure that the strong developer practice it has persists, and data quality remains a top priority.
Ramp has two distinct parts to its data platform: Snowflake, which runs dbt, and Looker, which runs both LookML transformations, as well as provides the visualization layer in the form of dashboards.
Given the high cadence in shipping new products, Ramp’s data team had to find a solution for improving build quality on an ongoing basis as well as allow the different stakeholders to establish better controls.
The team identified Foundational as the ideal partner to provide CI and preventative data quality for its data platform.
Foundational stands out in the complex world of data quality and lineage solution with our unique approach to preventative data quality, and enabling CI and downstream impact for every part of the data development lifecycle.
The technology built by Foundational allows for easy integration without any code changes needed, ensuring a quick onboarding process that covers every part of the stack.
Foundational’s source-code based lineage is unique, allowing it to analyze pending code changes as opposed to existing lineage and data quality solutions that are reactive, and limited to the data warehouse.
The Foundational team are world-class experts in reverse engineering and source code analysis, who put customer satisfaction and user value as the top priority.
Our expertise is not just in handling the tools and technology but also in understanding real-world deployments and constraints, being able to adapt to various challenges when analyzing complex data environments.
With Foundational, Ramp found a partner who allowed it to scale even faster with minimal concerns around data quality and shipping bad code.
Foundational partnered with Ramp’s Analytics Engineering team to execute a better CI experience that checks new code across Snowflake, dbt and Looker. Our holistic solution incorporated three core components:
Our integrations with dbt, Snowflake and Looker allowed Foundational to automate data lineage across all of these tools – being always updated to the latest commit. While traditional lineage tools rely on query logs, Foundational’s source-code based approach allows for real-time lineage, better accuracy, and improved visibility to non-warehouse flows.
Foundational’s integration with GitHub also allows for the Downstream Impact Analysis to get communicated through GitHub comments against every pull request, providing visibility to analytics engineers to the cross-platform impact of their code change, before deploying it. This visibility provides confidence and replaces the need for manual checks, ultimately costing time and money to Ramp.
Foundational allows Ramp to prevent far more data quality incidents than before. Prior to Foundational, Ramp relied on in-house CI checks to validate some aspects of the code, not really knowing the full extent of a given code change on cross-platform elements, for example in the case of a dbt change impacting Looker dashboards.
Foundational’s Data Quality checks introduce an additional layer of data quality that checks the code before its deployment, to identify potential data issues and prevent incidents.
With Foundational, Ramp was able to increase its daily build success rate from 85% to 95%+, an improvement of more than 10% in daily builds success rate, which translates to both time and cost savings.
The third component of our solution involved the ability for Ramp to create rules and policies against code changes that would impact data. This provides for better data governance, improves alignment, and further reduces the change for unexpected code changes which would impact mission-critical data assets.
Foundational uses a connector-based setup and does not process or extract any data or user records. This means that the only access we get is to code and configurations, which provides for better data security and access controls.
Foundational natively integrates with git, which in Ramp’s case is provided by GitHub, to power the developer experience directly in the interface developers use, regardless of their department in the organization.
The deployment was an overwhelming success, marked by clear improvements in KPIs and positive outcomes:
Most importantly, the client was satisfied. When asked about the integration, Kevin Chao, the Analytics Engineering Team Lead, said, “Foundational helps us improve the developer experience, ship code at a faster pace; Things that we previously worried about are no longer a concern. I really appreciate that everything is defined in code, that’s how we do things at Ramp with our own product. It’s a lot faster from a cycle time perspective.“
The success of this integration in helping data quality and cross-platform lineage against a complex, highly dynamic environment, is a strong testament to Foundational’s approach of looking directly at source-code as a new paradigm in data management and improving data quality.
As your organization considers different solutions to improving data quality, let Foundational become your partner to shift-left governance, preventative data quality, and automated data contracts. Our approach allows for minimal setup time, extended coverage, and improved developer experience when working with data.
