Modern Data

Master Data Starter

Match rules for customer, product, and supplier records, tested on your own data first.

GIST stepGroundIterate
Download the PDF

Get the Master Data Starter overview

A short PDF to share with your team. It covers what is in the package, how it works, and a worked example.

The problem

The same customer lives in five systems, five different ways.

Names are spelled differently, emails carry tags, phones come in four formats, and two different people share a name and a ZIP code. Most MDM programs spend months configuring match rules before anyone sees a single result.

What's included

Prove the match rules on your own records first.

01

Data models

Customer, product, supplier, and reference data, with golden records, cross-references, lineage, stewardship, and a change log. SQL models show how each one lands in your warehouse.

02

Match, merge, and survivorship rules

Rules live in plain configuration files. Comparators include exact, Jaro-Winkler, token set, name tokens, and numeric. Survivorship can take the most recent, most frequent, or most complete value, or follow source priority.

03

Stewardship queue

Pairs the engine is unsure about go to a data steward. The steward’s decision overrides the score on every later run.

04

Scorecards and data contracts

The evaluate command measures precision and recall against a labeled sample and fails CI below a floor you set. The explain command shows field-by-field scores for any pair. Each source gets a data contract template.

How it works

How matching works.

  1. Standardize

    Names folded and nicknames expanded, emails normalized, phones put into E.164, addresses and IDs cleaned.

  2. Block

    Only records that share a key, such as email, phone, or last name plus ZIP, get compared.

  3. Score

    Each field has a comparator and a weight. The score is the weighted average over the fields both records have.

  4. Classify

    A threshold sorts each pair into match, review, or no match. Hard rules win, such as never merging two different tax IDs.

  5. Review

    A steward decides the review pairs, and those decisions stick on every later run.

  6. Cluster and survive

    Matched records become one golden record. Survivorship picks each value, and lineage records where it came from.

Worked example

359 customer records from three systems.

Records from a CRM, an ERP, and an online store describe 150 real customers, with the mess real data has.

Seeded in the data
  • CRM duplicates in all caps, with no email
  • ERP names written “Last, First”, with nicknames
  • Store logins with +shop tags and dots in Gmail addresses
  • Phones in four formats, some mistyped
  • Two different James Smiths in the same ZIP code
Result

With the rules as shipped, the engine makes no false merges. Pairs with mistyped phones go to the steward queue, and the steward’s calls raise recall on the next run. The two James Smiths stay apart.

Where it fits

What it runs on, and where it stops.

Works with

  • Informatica MDM, Reltio, Profisee, Semarchy, and others
  • Custom builds in your warehouse
  • Spark for large volumes

Assumptions and limits

  • The built-in engine handles tuning and up to a few hundred thousand records. Larger volumes run on your MDM platform or Spark
  • Standardization follows US address and phone formats. Other countries need their own rules
  • Clustering is transitive, so the scorecard flags clusters for a person to check
Ask Dex

Ask about the Master Data Starter

Dex answers from our own pages and links to where it found the answer.

How many copies of each customer do you have?

Bring a sample of your messiest records. We will walk you through a match run on them.