MDM HUB

Source of Truth on Unity Catalog

People · Finance · Manufacturing

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Implementation Architecture & Operating Guide

Version 1.0  |  Confidential

1. Situation & Design Decisions

1.1 Current State

The organisation has three domain-specific Gold catalogs already running on the same Databricks workspace / metastore. Each catalog holds curated, high-quality data for its domain but operates in isolation with no shared entity keys across domains.

Domain	Existing Catalog	Key Entities	Key Problem
People	people_gold	Employee, Department, Role, Position	Employee IDs local to HR system — no link to Finance cost centres or Manufacturing plants
Finance	finance_gold	Cost Centre, GL Account, Budget, Vendor, Legal Entity	Vendor and employee records exist independently with no shared identifier
Manufacturing	manufacturing_gold	Plant, Asset, Work Order, BOM, Production Run	Plant codes and responsible persons not linked to People or Finance domains

1.2 Target State

A new Unity Catalog — mdm_hub — is created in the same Databricks metastore. It acts as the Single Source of Truth (SSoT) by:

• Pulling from all three existing Gold catalogs via live views (no data duplication).
• Running entity matching to generate a Global Entity Index (GEI) that assigns a stable mdm_id across domains.
• Publishing certified Golden Record tables that all Databricks workloads query instead of going to domain catalogs directly.
• Registering all assets in Azure Purview for lineage, classification, and stewardship.

1.3 Key Design Decisions

Decision	Choice	Rationale
Hub location	New Unity Catalog: mdm_hub	Clean separation — domain catalogs stay unchanged; Hub is additive only
Data movement	Views over domain Gold tables (no ETL copy)	Zero duplication, always current, reduces storage cost
Entity matching	Deterministic first, probabilistic fallback	No shared keys exist; matching engine generates mdm_id
Key strategy	Surrogate mdm_id (UUID) as global key	Domain keys preserved as source_id columns; mdm_id is the new universal identifier
Consuming pattern	Direct catalog reference mdm_hub.golden.*	Downstream notebooks / jobs switch catalog reference — no API layer needed initially
Governance	Azure Purview scans mdm_hub; Purview owns lineage + classification	Purview tracks Bronze→Gold→Hub lineage end-to-end

2. MDM Hub Architecture

2.1 Catalog & Schema Structure

The mdm_hub catalog is organised into four schemas, each serving a distinct layer of the hub:

Schema	Full Path	Purpose	Access
source_views	mdm_hub.source_views.*	Live views pointing to domain Gold tables — read-only projections, no transformation	MDM Engineers (read), Matching jobs (read)
matching	mdm_hub.matching.*	Entity matching output — candidate pairs, match scores, manual review queue	MDM Engineers (write), Stewards (review)
golden	mdm_hub.golden.*	Certified Golden Record tables — the single source of truth for all consuming workloads	All Databricks principals (read)
audit	mdm_hub.audit.*	Change history, match decision log, DQ metrics, steward actions	DGO, Stewards, Compliance (read)

2.2 Data Flow

Data flows through four stages from domain catalogs to consuming workloads:

Stage	From → To	Mechanism	Frequency
1 — Project	people_gold / finance_gold / manufacturing_gold → mdm_hub.source_views	Unity Catalog Views (CREATE VIEW … AS SELECT …)	Real-time (live views)
2 — Match	mdm_hub.source_views → mdm_hub.matching	Databricks notebook / DLT matching pipeline (blocking + scoring)	Daily batch or triggered on Gold update
3 — Certify	mdm_hub.matching (approved pairs) → mdm_hub.golden	MERGE INTO golden records; steward sign-off gate	Daily after matching run
4 — Consume	mdm_hub.golden → downstream notebooks, dashboards, jobs	Standard Unity Catalog SELECT; switch catalog in connection string	On-demand / streaming

2.3 Golden Record Entity Model

Each domain contributes specific entities to the Hub. The matching engine links them using a shared mdm_id:

Hub Entity	Source: People	Source: Finance	Source: Manufacturing	Link Logic
Person (Golden)	employee_id, name, email, department, role	cost_centre_owner_id, vendor_contact	plant_manager_id, asset_owner_id	Match on: name + email (deterministic); name + phone (probabilistic)
Organisational Unit	department_code, dept_name	cost_centre_code, legal_entity	plant_code, plant_name	Match on: dept_code ↔ cost_centre_code mapping table + name fuzzy
Location	(office location if present)	legal_entity address	plant address, geo coordinates	Match on: address normalisation + geo distance threshold
Vendor / Supplier	(not present)	vendor_id, vendor_name, tax_id	(not present)	Single source — dedup within Finance only

3. Entity Matching Design

Since no shared key exists across domain catalogs, a two-phase matching engine is required to generate the global mdm_id.

3.1 Matching Phases

Phase	Method	When Used	Output
Phase 1 — Deterministic	Exact match on email, tax_id, or known mapping table (dept_code ↔ cost_centre)	Where at least one reliable shared attribute exists	High-confidence match pairs (score = 1.0)
Phase 2 — Probabilistic	Jaro-Winkler name similarity + phonetic + address normalisation (via Databricks + difflib / Zingg OSS)	Remaining unmatched records after Phase 1	Scored candidate pairs (threshold ≥ 0.85 auto-approve, 0.70–0.85 manual review)
Phase 3 — Manual Review	Steward reviews low-confidence pairs in review UI (Purview / custom notebook widget)	Pairs scoring 0.70–0.85	Steward approve / reject → written to matching.review_decisions

3.2 Match Score Thresholds

Score Range	Decision	Action	Audit Trail
≥ 0.90	Auto-Approve — High Confidence	MERGE into mdm_hub.golden immediately	Written to audit.match_log with auto_approved=true
0.75 – 0.89	Manual Review Required	Held in matching.review_queue; steward action required	Written to audit.review_queue; SLA 48 hours
< 0.75	No Match — Reject	Records remain as standalone entities in their domain	Written to audit.match_log with rejected=true

3.3 Matching Pipeline Code (Databricks DLT)

The matching pipeline is implemented as a Databricks Delta Live Tables pipeline. Key stages:

# mdm_matching_pipeline.py  — DLT pipeline for Person entity matching

import dlt

from pyspark.sql import functions as F

from pyspark.sql.functions import col, udf, levenshtein, soundex

from pyspark.sql.types import FloatType

# ── STAGE 1: Source projections (already in source_views, referenced here) ──

@dlt.table(name=’people_candidates’, schema=’mdm_hub.matching’)

def people_candidates():

    return spark.read.table(‘mdm_hub.source_views.vw_people_person’) \

           .select(’employee_id’,’full_name’,’email’,’phone’,’dept_code’) \

           .withColumn(‘source’, F.lit(‘people’))

@dlt.table(name=’finance_candidates’, schema=’mdm_hub.matching’)

def finance_candidates():

    return spark.read.table(‘mdm_hub.source_views.vw_finance_vendor_contact’) \

           .select(‘contact_id’,’full_name’,’email’,’phone’,’cost_centre’) \

           .withColumn(‘source’, F.lit(‘finance’))

# ── STAGE 2: Blocking (reduce comparison space) ──

# Block on first 3 chars of surname + soundex of email domain

@dlt.table(name=’match_candidates_blocked’)

def blocked():

    p = dlt.read(‘people_candidates’).withColumn(‘block_key’,

            F.concat(F.substring(‘full_name’, 1, 3), soundex(’email’)))

    f = dlt.read(‘finance_candidates’).withColumn(‘block_key’,

            F.concat(F.substring(‘full_name’, 1, 3), soundex(’email’)))

    return p.join(f, ‘block_key’, ‘inner’) \

            .filter(col(‘p.source’) != col(‘f.source’))

# ── STAGE 3: Score pairs ──

@udf(FloatType())

def jaro_winkler(s1, s2):

    # install jellyfish on cluster: pip install jellyfish

    import jellyfish

    return float(jellyfish.jaro_winkler_similarity(s1 or ”, s2 or ”))

@dlt.table(name=’scored_pairs’)

def scored_pairs():

    blocked = dlt.read(‘match_candidates_blocked’)

    return blocked.withColumn(‘name_score’, jaro_winkler(‘p.full_name’,’f.full_name’)) \

                  .withColumn(’email_exact’, (col(‘p.email’) == col(‘f.email’)).cast(‘float’)) \

                  .withColumn(‘final_score’,

                      col(’email_exact’) * 0.6 + col(‘name_score’) * 0.4)

# ── STAGE 4: Route by threshold ──

@dlt.table(name=’auto_approved_pairs’)

@dlt.expect_or_drop(‘high_confidence’, ‘final_score >= 0.90’)

def auto_approved():

    return dlt.read(‘scored_pairs’).filter(col(‘final_score’) >= 0.90)

@dlt.table(name=’review_queue’)

def review_queue():

    return dlt.read(‘scored_pairs’) \

              .filter((col(‘final_score’) >= 0.75) & (col(‘final_score’) < 0.90)) \

              .withColumn(‘review_status’, F.lit(‘PENDING’)) \

              .withColumn(‘assigned_steward’, F.lit(None))

3.4 Golden Record MERGE Logic

After matching, the golden record is created or updated using a MERGE statement. The mdm_id is a UUID generated on first creation and preserved on subsequent updates.

— mdm_hub.golden.person — MERGE from approved pairs

MERGE INTO mdm_hub.golden.person AS target

USING (

  SELECT

    COALESCE(existing.mdm_id, uuid()) AS mdm_id,

    p.employee_id     AS people_source_id,

    f.contact_id      AS finance_source_id,

    COALESCE(p.full_name, f.full_name)  AS canonical_name,

    COALESCE(p.email,    f.email)       AS canonical_email,

    COALESCE(p.phone,    f.phone)       AS canonical_phone,

    p.dept_code       AS people_dept_code,

    f.cost_centre     AS finance_cost_centre,

    current_timestamp()                 AS last_updated,

    ‘AUTO_MATCH’                        AS update_source

  FROM mdm_hub.matching.auto_approved_pairs ap

  JOIN mdm_hub.source_views.vw_people_person  p ON ap.people_id  = p.employee_id

  JOIN mdm_hub.source_views.vw_finance_vendor_contact f ON ap.finance_id = f.contact_id

  LEFT JOIN mdm_hub.golden.person existing

         ON existing.people_source_id = p.employee_id

) AS source

ON target.people_source_id = source.people_source_id

   OR target.finance_source_id = source.finance_source_id

WHEN MATCHED THEN UPDATE SET

    target.finance_source_id = source.finance_source_id,

    target.canonical_email   = source.canonical_email,

    target.last_updated      = source.last_updated

WHEN NOT MATCHED THEN INSERT *;

4. mdm_hub Catalog — Full Implementation

4.1 Catalog Setup (Databricks SQL)

— Step 1: Create the hub catalog

CREATE CATALOG IF NOT EXISTS mdm_hub

  COMMENT ‘Master Data Management Hub — Single Source of Truth’;

— Step 2: Create schemas

CREATE SCHEMA IF NOT EXISTS mdm_hub.source_views COMMENT ‘Live views over domain Gold catalogs’;

CREATE SCHEMA IF NOT EXISTS mdm_hub.matching      COMMENT ‘Entity matching staging and review’;

CREATE SCHEMA IF NOT EXISTS mdm_hub.golden        COMMENT ‘Certified Golden Records’;

CREATE SCHEMA IF NOT EXISTS mdm_hub.audit         COMMENT ‘Change history, DQ metrics, steward log’;

— Step 3: Grant read access on Golden layer to all domain principals

GRANT USE CATALOG ON CATALOG mdm_hub TO `data-consumers`;

GRANT USE SCHEMA  ON SCHEMA mdm_hub.golden TO `data-consumers`;

GRANT SELECT      ON ALL TABLES IN SCHEMA mdm_hub.golden TO `data-consumers`;

— Step 4: Restrict matching + audit to MDM engineers and stewards only

GRANT USE SCHEMA ON SCHEMA mdm_hub.matching TO `mdm-engineers`;

GRANT USE SCHEMA ON SCHEMA mdm_hub.audit    TO `mdm-engineers`, `data-stewards`;

4.2 Source Views — Domain Projections

Views in mdm_hub.source_views are thin projections — they rename columns to a canonical standard and drop domain-internal fields that are not needed in the hub.

— People domain view

CREATE OR REPLACE VIEW mdm_hub.source_views.vw_people_person AS

SELECT

    employee_id                          AS source_id,

    CONCAT(first_name, ‘ ‘, last_name)   AS full_name,

    LOWER(TRIM(work_email))              AS email,

    mobile_number                        AS phone,

    department_code                      AS dept_code,

    job_title                            AS role,

    employment_status,

    ‘people_gold’                        AS source_catalog,

    _updated_at                          AS source_updated_at

FROM people_gold.hr.employees

WHERE employment_status = ‘ACTIVE’;

— Finance domain view

CREATE OR REPLACE VIEW mdm_hub.source_views.vw_finance_cost_centre AS

SELECT

    cost_centre_code                     AS source_id,

    cost_centre_name,

    legal_entity_code,

    owner_employee_ref                   AS finance_owner_ref,  — link candidate to People

    gl_account_group,

    ‘finance_gold’                       AS source_catalog,

    _updated_at                          AS source_updated_at

FROM finance_gold.gl.cost_centres

WHERE active_flag = 1;

— Manufacturing domain view

CREATE OR REPLACE VIEW mdm_hub.source_views.vw_manufacturing_plant AS

SELECT

    plant_code                           AS source_id,

    plant_name,

    plant_manager_employee_ref,          — link candidate to People

    site_address,

    region_code,

    ‘manufacturing_gold’                 AS source_catalog,

    _updated_at                          AS source_updated_at

FROM manufacturing_gold.operations.plants

WHERE operational_status = ‘ACTIVE’;

4.3 Golden Record Table DDL

— PERSON — core cross-domain entity

CREATE TABLE IF NOT EXISTS mdm_hub.golden.person (

    mdm_id               STRING    NOT NULL COMMENT ‘Global surrogate key — stable UUID’,

    people_source_id     STRING    COMMENT ’employee_id from people_gold’,

    finance_source_id    STRING    COMMENT ‘contact_id from finance_gold’,

    mfg_source_id        STRING    COMMENT ‘manager ref from manufacturing_gold’,

    canonical_name       STRING    NOT NULL,

    canonical_email      STRING,

    canonical_phone      STRING,

    dept_code            STRING    COMMENT ‘From People domain’,

    cost_centre_code     STRING    COMMENT ‘From Finance domain’,

    plant_code           STRING    COMMENT ‘From Manufacturing domain’,

    match_confidence     FLOAT     COMMENT ‘Score from matching engine’,

    certification_status STRING    DEFAULT ‘CERTIFIED’,

    certified_by         STRING,

    certified_at         TIMESTAMP,

    last_updated         TIMESTAMP,

    PRIMARY KEY (mdm_id) NOT ENFORCED

) USING DELTA

TBLPROPERTIES (

    ‘delta.enableChangeDataFeed’ = ‘true’,

    ‘quality’ = ‘gold’,

    ‘domain’  = ‘cross-domain’,

    ‘owner’   = ‘mdm-team’

);

— ORGANISATIONAL UNIT

CREATE TABLE IF NOT EXISTS mdm_hub.golden.org_unit (

    mdm_id               STRING    NOT NULL,

    people_dept_code     STRING,

    finance_cost_centre  STRING,

    mfg_plant_code       STRING,

    canonical_name       STRING    NOT NULL,

    legal_entity_code    STRING,

    region_code          STRING,

    head_person_mdm_id   STRING    COMMENT ‘FK to mdm_hub.golden.person’,

    match_confidence     FLOAT,

    last_updated         TIMESTAMP,

    PRIMARY KEY (mdm_id) NOT ENFORCED,

    FOREIGN KEY (head_person_mdm_id) REFERENCES mdm_hub.golden.person(mdm_id) NOT ENFORCED

) USING DELTA;

— LOCATION

CREATE TABLE IF NOT EXISTS mdm_hub.golden.location (

    mdm_id               STRING    NOT NULL,

    mfg_plant_code       STRING,

    finance_entity_addr  STRING,

    canonical_address    STRING,

    region_code          STRING,

    country_code         STRING,

    geo_lat              DOUBLE,

    geo_lng              DOUBLE,

    org_unit_mdm_id      STRING,

    last_updated         TIMESTAMP,

    PRIMARY KEY (mdm_id) NOT ENFORCED

) USING DELTA;

5. Azure Purview Integration

5.1 Purview Registration & Scanning

Step	Action	Detail
1	Register Databricks source in Purview	Purview → Data Map → Register → Azure Databricks. Point to the workspace. Purview authenticates via managed identity.
2	Create Scan targeting mdm_hub catalog	Scan filter: catalog = mdm_hub, schemas = source_views, matching, golden, audit. Schedule: daily.
3	Apply Classification Rules	Create custom classification rules in Purview for: MDM_GOLDEN_RECORD (tag all tables in golden.*), PII (email, phone, name columns), CROSS_DOMAIN_KEY (mdm_id column).
4	Configure Business Glossary	Create terms in Purview Glossary: Golden Record, mdm_id, Source of Truth, Match Confidence, Domain Catalog. Link terms to mdm_hub.golden.* table columns.
5	Lineage wiring	Purview auto-captures Unity Catalog lineage. Manually register the DLT matching pipeline as a lineage process: source_views → matching → golden.
6	Data Policy (optional)	Create Purview Data Access Policy to allow data-consumers group SELECT on mdm_hub.golden.* — pushed back to Unity Catalog via policy sync.

5.2 Lineage Map (Purview)

End-to-end lineage visible in Azure Purview after setup:

Lineage Node	Type	Catalog / System	Connected To
people_gold.hr.employees	Source Table	people_gold (Databricks)	mdm_hub.source_views.vw_people_person
finance_gold.gl.cost_centres	Source Table	finance_gold (Databricks)	mdm_hub.source_views.vw_finance_cost_centre
manufacturing_gold.operations.plants	Source Table	manufacturing_gold (Databricks)	mdm_hub.source_views.vw_manufacturing_plant
DLT Matching Pipeline	Process / ETL	Databricks DLT	source_views → matching.scored_pairs
mdm_hub.matching.auto_approved_pairs	Intermediate Table	mdm_hub (Databricks)	golden.person (via MERGE job)
mdm_hub.golden.person	Golden Record Table	mdm_hub (Databricks)	All downstream consuming notebooks/jobs
mdm_hub.golden.org_unit	Golden Record Table	mdm_hub (Databricks)	All downstream consuming notebooks/jobs

6. Data Governance Framework for the Hub

6.1 Governance Principles (Hub-Specific)

Principle	What It Means for mdm_hub
Domain catalogs are read-only inputs	mdm_hub never writes back to people_gold, finance_gold, or manufacturing_gold. Data flows one direction only: domain → hub.
mdm_id is immutable once assigned	A golden record’s mdm_id never changes. If a merge decision is reversed, the record is marked RETIRED — not deleted.
Match decisions are auditable	Every match approval (auto or manual) is written to mdm_hub.audit.match_log with timestamp, steward ID, and score.
Golden = Certified	A record in mdm_hub.golden.* is certified by definition. Uncertified records live in matching.review_queue only.
No consumer touches matching or audit schemas	Only MDM engineers and stewards access matching.* and audit.*. Consumers only see golden.*.

6.2 Data Quality Gates

Records are blocked from promotion to golden.* unless they pass all quality gates in the DLT pipeline:

Gate	Rule	Threshold	Action on Failure
Completeness	canonical_name must not be null	100%	Record dropped — written to matching.rejected
Email validity	canonical_email must match regex ^[\w.-]+@[\w.-]+\.[a-z]{2,}$	Soft check	Flag set; record still promoted but tagged EMAIL_SUSPECT
Match score	auto-approval requires final_score ≥ 0.90	90%	Routed to review_queue if 0.75–0.89; rejected if < 0.75
Duplicate check	No two golden records may share the same canonical_email	0 duplicates	Second record flagged DUPLICATE_SUSPECT; steward review
Cross-domain link	At least 2 source_ids populated for entity to be ‘cross-domain certified’	≥ 2 sources	Single-source records marked SINGLE_SOURCE (valid but not cross-domain)

6.3 Governance Structure

Role	Responsibility in Hub Context	Catalog Access
MDM Hub Owner (1)	Accountable for mdm_hub catalog; approves architecture changes; chairs MDM Working Group	Metastore admin on mdm_hub
Domain Data Steward — People (1)	Reviews person match queue; approves/rejects low-confidence pairs involving People records	mdm_hub.matching + mdm_hub.audit (read/write review_queue)
Domain Data Steward — Finance (1)	Reviews cost centre and org unit matches involving Finance records	mdm_hub.matching + mdm_hub.audit (read/write review_queue)
Domain Data Steward — Manufacturing (1)	Reviews plant and location matches involving Manufacturing records	mdm_hub.matching + mdm_hub.audit (read/write review_queue)
MDM Engineer (2)	Maintains DLT pipelines, matching rules, golden DDL, Purview connector	Full access to mdm_hub (all schemas)
Data Consumer	Reads from mdm_hub.golden.* for analytics, reporting, downstream jobs	mdm_hub.golden.* SELECT only

7. Target Operating Model

7.1 Daily Operating Rhythm

Time	Activity	Trigger	Owner
00:00 – 02:00	Domain Gold tables refreshed by existing domain pipelines	Scheduled domain jobs	Domain Engineers
02:00 – 04:00	DLT matching pipeline runs: source_views → matching.*	Databricks workflow schedule	MDM Engineer (automated)
04:00 – 06:00	Auto-approved pairs MERGED into golden.*	Post-matching workflow trigger	MDM Engineer (automated)
08:00	Stewards notified of new review_queue items via email alert	Databricks SQL alert on review_queue count	MDM Engineer (automated alert)
08:00 – 12:00	Stewards process review queue; approve/reject pairs	Steward manual action in Purview / notebook UI	Domain Stewards
12:00	Steward-approved pairs MERGED into golden.* (second daily run)	Post-review workflow trigger	MDM Engineer (automated)
18:00	Azure Purview incremental scan on mdm_hub	Purview scheduled scan	Purview (automated)
23:00	DQ metrics computed; alerts sent if golden.* thresholds breached	Databricks SQL alert	MDM Engineer (automated)

7.2 Engagement Model

Team	How They Engage with the Hub	Channel
People / HR Team	Steward reviews person match queue; approves merges; raises data issues via Jira MDM board	Purview Stewardship portal + Jira
Finance Team	Steward reviews cost centre / org unit matches; validates canonical legal entity names	Purview + Databricks SQL notebook
Manufacturing Team	Steward reviews plant / location matches; validates plant manager links	Purview + Databricks SQL notebook
BI / Analytics Consumers	Switch Databricks SQL catalog reference from domain catalog to mdm_hub.golden — query golden records directly	Databricks SQL / notebook
Data Engineering (downstream)	Jobs reference mdm_hub.golden.person.mdm_id as foreign key for cross-domain joins	Unity Catalog table reference
Platform / DevOps	CI/CD deploys matching rule changes via Databricks Asset Bundles (DAB); no manual cluster changes	GitHub Actions + DAB

7.3 Change Control

• Changes to matching rules (score weights, blocking keys) require a pull request reviewed by MDM Hub Owner and at least one Domain Steward.
• Schema changes to mdm_hub.golden.* require impact assessment (check downstream notebook dependencies via Purview lineage) and a 2-week notice to consumers.
• New domain onboarding (e.g. adding a Procurement catalog) follows a 6-week intake process: source view → matching config → golden DDL → steward assignment → go-live.

8. RACI Matrix

R = Responsible  |  A = Accountable  |  C = Consulted  |  I = Informed

8.1 MDM Hub Activities

Activity	MDM Hub Owner	People Steward	Finance Steward	Mfg Steward	MDM Engineer	Consumer	DGO
Design mdm_hub catalog structure	A	C	C	C	R	I	C
Create/maintain source_views	C	C	C	C	R/A	I	I
Design matching rules (Person)	A	R	C	I	C	I	C
Design matching rules (Org Unit)	A	C	R	C	C	I	C
Design matching rules (Location)	A	I	C	R	C	I	C
Build DLT matching pipeline	C	I	I	I	R/A	I	I
Review match queue — Person	I	R/A	I	I	I	I	I
Review match queue — Org Unit	I	C	R/A	C	I	I	I
Review match queue — Location	I	I	C	R/A	I	I	I
Certify Golden Records	A	C	C	C	R	I	C
Grant consumer access to golden.*	A	I	I	I	R	I	C
Monitor DQ metrics daily	I	I	I	I	R/A	I	I
Resolve match pipeline failures (P1)	A	I	I	I	R	I	I
Purview scan & lineage config	C	I	I	I	R/A	I	C
Quarterly access review	A	C	C	C	R	I	C
Onboard new domain to hub	A	C	C	C	R	I	C
Retire / archive golden record	A	C	C	C	R	I	C
Report Hub KPIs to leadership	R/A	C	C	C	C	I	C

9. Hub KPIs & Monitoring

KPI	Target	Frequency	Source	Owner
Golden Record Coverage — Person	≥ 95% of active employees matched across ≥ 2 domains	Weekly	mdm_hub.audit.match_log	People Steward
Golden Record Coverage — Org Unit	≥ 90% of dept codes linked to cost centres	Weekly	mdm_hub.golden.org_unit	Finance Steward
Auto-approval Rate	≥ 80% of candidate pairs auto-approved (score ≥ 0.90)	Daily	mdm_hub.matching.scored_pairs	MDM Engineer
Review Queue SLA	0 items in review_queue older than 48 hours	Daily	mdm_hub.matching.review_queue	MDM Hub Owner
Golden Record Completeness	canonical_name populated: 100%; canonical_email: ≥ 95%	Daily	mdm_hub.golden.person	MDM Engineer
Duplicate Golden Records	0 duplicates in golden.* (same email, different mdm_id)	Daily	DQ check job on golden.*	MDM Engineer
Purview Lineage Coverage	100% of golden.* tables have end-to-end lineage in Purview	Monthly	Azure Purview Insights	DGO
Consumer Adoption	≥ 80% of cross-domain notebooks referencing mdm_hub.golden.* (not domain catalogs)	Monthly	Unity Catalog query audit log	MDM Hub Owner

10. Implementation Plan

Week	Activity	Deliverable	Owner
Week 1–2	Create mdm_hub catalog; all 4 schemas; RBAC grants; register in Azure Purview	mdm_hub catalog live; Purview scan running	MDM Engineer
Week 2–3	Build source_views for all 3 domain catalogs; validate column mapping with stewards	9 source views live; steward sign-off on canonical column names	MDM Engineer + Stewards
Week 3–5	Build DLT matching pipeline for Person entity; unit test with sample data in sandbox	Matching pipeline in staging; scored_pairs table populated	MDM Engineer
Week 5–6	Stewards define match rule thresholds; configure review_queue alerts	Threshold config agreed; alert triggers tested	Stewards + MDM Engineer
Week 6–7	First full matching run on Person; stewards clear review queue; first golden.person load	golden.person table with first certified records	MDM Engineer + People Steward
Week 7–9	Extend matching to Org Unit; build golden.org_unit; link to golden.person via head_person_mdm_id	golden.org_unit live; cross-domain links established	MDM Engineer + Finance + Mfg Stewards
Week 9–10	Build golden.location; wire full Purview lineage; configure DQ monitoring jobs	All 3 golden tables live; Purview lineage end-to-end; DQ dashboard in Power BI	MDM Engineer
Week 10–11	Consumer onboarding — update downstream notebooks to reference mdm_hub.golden.*	Consuming jobs switched; adoption metric baseline set	MDM Engineer + consumers
Week 11–12	Hypercare — daily steward + engineer review; KPI baselining; documentation handoff	Hub in steady-state operations; runbook complete	MDM Hub Owner

10.1 Critical Dependencies

• Domain Gold tables must have stable, documented schemas before source_views are built. Any upstream schema change breaks the view.
• At least one reliable overlap attribute (email, employee_ref column in Finance and Manufacturing) must exist for deterministic matching to work. Validate with stewards in Week 2.
• Azure Purview Databricks connector requires the workspace to be on Unity Catalog (not legacy Hive metastore). Confirm metastore type before Week 1.
• Stewards from all three domains must be assigned and available by Week 3 for threshold configuration — without this, matching cannot go live.