---
title: "Data Platform Foundation Accelerator — Datagist"
description: "A governed lakehouse in Terraform for Azure Databricks: private networking, Unity Catalog grants, quality rules, and security checks on every pull request."
url: https://datagist.ai/accelerators/data-platform-foundation
---

[Accelerators](https://datagist.ai/accelerators) Data Platform Foundation

[Modern Data](https://datagist.ai/services/modern-data) 

# Data Platform Foundation

A lakehouse defined in Terraform, with access rules and quality checks from the first deploy.

GIST step [Iterate](https://datagist.ai/approach#iterate) [Ship](https://datagist.ai/approach#ship)

 Download the PDF 

## Get the Data Platform Foundation overview

A short PDF to share with your team. It covers what is in the package, how it works, and a worked example.

On this page The problem What's included How it works Worked example Where it fits

The problem 

## The first months of a data program go into plumbing.

Networking, storage, identity, access policies, and deployment pipelines all have to be right before anyone ships a data product. Teams rebuild them for every program, and the security settings get decided last.

What's included 

## A governed lakehouse, defined in Terraform.

01 

### Reference architecture

Built for Azure first, with reference designs for AWS and Google Cloud and variants for Microsoft Fabric, Snowflake, and BigQuery. It includes an access model for admins, engineers, and analysts.

02 

### Terraform modules

Network, storage, Key Vault, Databricks, Unity Catalog, and monitoring. Dev and prod have separate settings, each with its own remote state.

03 

### Ingestion jobs

A Databricks Asset Bundle with templates for files through Auto Loader, incremental database loads over JDBC with a watermark, and paginated REST APIs. Data lands in bronze and is typed and tested on its way to silver and gold.

04 

### Quality rules and CI/CD

Quality rules in YAML send bad rows to a quarantine table and stop a load that crosses a set threshold. Production deploys log in with OIDC and wait for an approval.

How it works 

## What gets built on Azure.

1. ### Private network  
A virtual network with private endpoints, delegated Databricks subnets, and private DNS zones.
2. ### Governed lake  
ADLS Gen2 with one container per layer: landing, bronze, silver, and gold. Shared keys and public access are switched off.
3. ### Secrets  
Key Vault with role-based access only, network deny by default, and purge protection in production.
4. ### Compute  
A Premium Databricks workspace with VNet injection and no public IPs. Production gets a private front end.
5. ### Access as code  
Unity Catalog grants. Admins get everything, engineers read and write, and analysts read silver and gold.
6. ### Audit logs  
Storage, Key Vault, and Databricks audit logs go to Log Analytics.

Worked example 

## A weakened security setting fails in review.

Guardrail checks run on every pull request and need no cloud access, so problems show up while the change is still a diff.

What the checks cover 
- Wiring between the network, storage, and compute modules
- Environment settings files, complete and consistent
- Security settings from the reference design
- Quality rules on every source load

Result 

A change that loosens a security setting fails before it reaches a subscription. Production deployments still wait for a person to approve them.

Where it fits 

## What it runs on, and where it stops.

### Works with

- Azure first: ADLS Gen2, Azure Databricks, Unity Catalog, and Key Vault
- Reference designs for AWS and Google Cloud
- Microsoft Fabric, Snowflake, and BigQuery variants

### Assumptions and limits

- The static checks pass today. Validating against your own subscription is the first job on your side
- One environment per state file. Account-level resources are set up once
- Databricks, storage, and logging cost money to run, so we size them with you first

Related accelerators 

[Master Data Starter Golden records land in its silver and gold layers. View details of the Master Data Starter](https://datagist.ai/accelerators/mdm-starter) [Metrics Layer Starter Certified metrics read from its gold layer. View details of the Metrics Layer Starter](https://datagist.ai/accelerators/metrics-layer) [MLOps Templates Models train and score on its data. View details of the MLOps Templates](https://datagist.ai/accelerators/mlops-templates)

## Starting a lakehouse on Azure?

Show us your landing zone and network layout. We will show you where the modules plug in.

Download the PDF [Talk to us ](https://datagist.ai/contact)
