Anatomy · 06 · Reference design · in build
Anatomy of a data platform, one row at a time.
- CDC simulated with events
- ACID per table, per commit
- Spark in local mode
A reference lakehouse on Apache Iceberg with synthetic data: orders from an API and customers from MongoDB, followed through six pipeline runs.
each disc is one run's commit to that layer · open any database to see its rows
1Sources
2Bronze append-only
!bronze.rejects
3Silver dedup + MERGE
4Gold KPIs
Iceberg catalog
What the six runs show
Need a platform like this? WhatsApp →
Reference design with synthetic data (100 orders, 50 customers), not client work. Storage and catalog are a configuration profile: S3-compatible storage locally; a profile for Amazon S3 with a Glue or S3 Tables Iceberg catalog is documented, not deployed.