Medallion Architecture: The 3-Layer Pattern You Need Before Building a Data Pipeline

A real problem

Imagine you work at an e-commerce company. Data lives in many places: transactions in the app database, user clicks in event tracking, payments from a payment gateway, product data from inventory.

Then problems start:

• Analyst A uses data straight from the database. Analyst B uses a CSV export from last month. Their sales numbers differ, and both believe they are right.
• Raw data has duplicates, typos, and failed transactions that still get counted.
• One pipeline breaks, and you cannot rerun the process because the raw data was already overwritten.

Medallion Architecture is used to solve problems like these.

What is Medallion Architecture?

Medallion Architecture is a design pattern for organizing data flow in three layers. The names follow medal quality: Bronze, Silver, and Gold.

The core idea is simple: the business should not use raw data directly. Data is cleaned and improved in steps. At the end you have one clean source of truth that is ready to use.

Loading diagram...

Databricks popularized this idea. Many teams now use it, and it is not tied to one platform.

Three layers: what each one does

Bronze Layer: raw data

Bronze stores data exactly as it arrives from the source. No changes. What you ingest is what you keep.

• The format is often kept as-is (JSON, CSV, and similar), or written to a modern table format.
• Append-only: you only add data. You do not edit or delete it. Old records stay.
• Do not put business transforms here.

This layer is insurance. If a pipeline above Bronze fails, you do not need to ask another team for the data again, and you do not need to extract again from the production database. You replay from Bronze: rerun the transforms on data that is already stored.

Terms you will hear at work

Some companies call this layer Landing or Staging. If a teammate says "data staging area", they often mean the same thing as Bronze or the Raw layer.

Silver Layer: clean data

This is where cleaning happens. Raw data is transformed so it is structured, clean, and consistent.

Common work in Silver:

• Deduplication: remove duplicate records, for example when event tracking sends the same data twice.
• Standardization: IDR, Rp, and rupiah become one format; timestamps are aligned to UTC.
• Data quality checks: drop or flag invalid records, for example an email with no @ or a negative age.
• Join across tables, for example orders with customer data.
• PII masking: hide sensitive data such as national ID or credit card numbers.

Silver is the single source of truth for detailed data: one agreed version that people treat as the reference. If the debate is "which number is correct", look at Silver.

Terms you will hear at work

This cleaning is often called cleansing, conforming (making formats match across sources), or harmonization. The layer is sometimes called the Cleansed layer.

Gold Layer: data ready to consume

Gold is the business-level layer. Data is aggregated and modeled for consumers (dashboards, reports, machine learning).

Example Gold tables, named after the business question:

• daily_revenue_per_region
• customer_lifetime_value
• monthly_active_users

Some teams model Gold as a star schema: fact tables for events or transactions (fact_orders), dimension tables for context such as customers or products (dim_customer). That is a pattern for how tables relate. It does not replace the descriptive names above. The point is that Gold is shaped so it is easy to consume.

Terms you will hear at work

This layer is often called the Curated layer or Mart layer. A data mart is a set of tables for one business need, for example sales.

That set usually lives in a schema or a database. Teams often use these two words for the same container, depending on the tool (Spark, Glue, and Iceberg catalogs usually say database or namespace; Postgres and Snowflake keep database and schema as separate levels). Example: schema mart_sales, table daily_revenue_per_region. In SQL it looks like mart_sales.daily_revenue_per_region. The dot separates the container and the table. It is not part of the table name.

Inside the same mart you can still have descriptive tables or fact/dimension pairs.

Example flow: e-commerce transactions

Follow one dataset from source to consumers.

Bronze. Transactions are extracted from the app database every hour and stored as-is. This includes failed transactions, duplicates, and columns with mixed formats.

Silver.

• Remove duplicates using transaction_id.
• Drop transactions with status FAILED.
• Align column names: txn_amt becomes amount, and currency format is standardized.
• Join the customer table to add demographic fields.

Gold.

• Aggregate by day and region into the daily_revenue table.
• That table is ready for an executive dashboard.

The result: if a data engineer finds a bug in the aggregation, they fix the logic and reprocess from Silver. They do not need to touch the source, and they do not need to panic.

Why three layers, not one?

Problem without Medallion	What Medallion gives you
Raw data is overwritten during transform	Bronze is append-only, so you can always replay
Each team has its own "version of the truth"	Silver is the single source of truth
Complex transforms are mixed in one place	Each layer has one responsibility
A bug in one step breaks everything	You can fix one layer without touching the rest

This is separation of concerns, like splitting presentation, business logic, and data access in software engineering. Here you split levels of data maturity instead.

Example mapping on AWS

The idea is not tied to one platform. On AWS, one common stack is:

• Ingest into Bronze: AWS Glue, Amazon Kinesis, or Amazon DMS. DMS is often used for CDC (Change Data Capture): copy inserts, updates, and deletes from the source database without extracting the full table every time.
• Storage: Amazon S3, usually one bucket with prefixes bronze/, silver/, gold/, or separate buckets.
• Transform to Silver and Gold: AWS Glue (Spark), Amazon Athena, or dbt.
• Table format: Apache Iceberg or Delta Lake. Both add transaction guarantees (ACID) and a way to look at older data versions (time travel) on top of S3. You do not need those details yet to understand the pattern.
• Consume Gold: Amazon QuickSight for BI, SageMaker for ML, or query with Athena.

On other platforms (Databricks, Snowflake, GCP, Azure) the tool names change. The pattern stays the same. That is why the concept is worth more than memorizing tools.

Anti-patterns (what to avoid)

These show up often at work, and they are worth avoiding:

• Silver becomes a dumping ground. All joins and business rules land in Silver. That layer is hard to maintain, and Gold has no clear role. Keep Silver for cleansing and conforming. Put dashboard or KPI rules in Gold.
• Gold has too many tables and no standard. Hundreds of small tables, and nobody knows which number is official. Use consistent names, and decide which tables reports may use.
• Forcing three layers on a small case. One source and one dashboard do not need the full pattern. You only add complexity (over-engineering), not value. Match the design to the scale.
• Loose quality checks in Silver. Bronze looks neatly stored, but dirty data still moves up to Gold. If Silver does not filter well, the garbage only changes place.

When Medallion fits (and when it does not)

A good fit when:

• You have many data sources and many consumers.
• Data quality varies.
• You need audit or replay.

A weaker fit for a small project: one source, one dashboard. The full pattern adds complexity without much value.

Summary

• Medallion uses 3 layers: Bronze (raw, append-only), Silver (clean, single source of truth), Gold (aggregated, ready to consume).
• The main value is split responsibilities, audit and replay, plus one version of the truth.
• Terms you will hear often: raw, staging, cleansing, conforming, curated, mart, fact and dimension, CDC.
• Tools change. The pattern stays. Learn the idea, then adapt it on any platform.

If you are new to data engineering, Medallion Architecture is a foundation you will see often. Many mid-size and larger data teams use a variation of this pattern.

References

• What is the medallion lakehouse architecture? (Databricks)
• What is Medallion Architecture? (Databricks)