Data quality rules using dimensions

Nhập môn Chất lượng Dữ liệu

Chrissy Bloom

Head of Enterprise Data Strategy & Governance

Data quality rules

Data quality rule: type of business rule that validates whether data meets business requirements

Can be defined at:
- Dataset level
- Data element level

table depicting dataset and data element data quality rules

Completeness data quality rule

Dataset data quality rule: All expected records from the source must be loaded in the target table.

table depicting completeness data quality rule

Data element data quality rule: All records must have a Customer ID, Customer Name, and Customer Account Type populated.

table depicting completeness data quality rule

Timeliness data quality rule

Dataset data quality rule: All records in the customer dataset must be loaded by the 9:00 am.

table depicting timeliness data quality rule

Data element data quality rule: All records must have a tax ID populated by first time the customer's account status is "Open".

table depicting timeliness data quality rule

Validity data quality rule

Data element data quality rule:

All records must have a Birth Date value in the format MM/DD/YYY and the value must be in the past.
All records must have an Account Status of Open, Closed, or Pending.

table depicting validity data quality rule

Consistency data quality rule

Dataset data quality rule: The count of records loaded today must be within +/- 5% of the count of records loaded yesterday.

table depicting consistency data quality rule

Data element data quality rule: All Customer ID values in the AccountTable must also be present in the CustomerTable.

table depicting consistency data quality rule

Accuracy data quality rule

Data element data quality rule: All records in the CustomerTable must have accurate Customer Name, Customer Birthdate, and Customer Address fields when compared to the Tax Form.

table depicting accuracy data quality rule

tax form used for accuracy comparison

Uniqueness data quality rule

Data element data quality rule:

All records must have a unique Customer ID.
All records must have a unique combination of customer name, customer birth date, and customer address fields.

table depicting uniqueness data quality rule

Let's practice!

Nhập môn Chất lượng Dữ liệu