Update Expectation Suites

Introduction to Data Quality with Great Expectations

Davina Moossazadeh

Data Scientist

Adding Expectations

expectation = gx.expectations.ExpectTableColumnCountToEqual(
    value=10
)
suite = gx.ExpectationSuite(
    name="my_suite"
)
# Add Expectation to Suite
suite.add_expectation(
    expectation=expectation
)
Introduction to Data Quality with Great Expectations

Multiple Expectation Suites

# Create another Expectation Suite
another_suite = gx.ExpectationSuite(name="my_other_suite")
# Add the same Expectation to the new Suite
another_suite.add_expectation(expectation=expectation)

Expectations cannot belong to multiple Suites at once:

RuntimeError: Cannot add Expectation because it already belongs to an 
ExpectationSuite. If you want to update an existing Expectation, please call 
Expectation.save(). If you are copying this Expectation to a new ExpectationSuite, 
please copy it first (the core expectations and some others support 
copy(expectation)) and set `Expectation.id = None`.
Introduction to Data Quality with Great Expectations

Copying Expectations

If you are copying this Expectation to a new ExpectationSuite, please copy it first 
(the core expectations and some others support copy(expectation)) and set 
`Expectation.id = None`.

Copy the Expectation, set its .id to None, and add it to the new Suite without errors:

expectation_copy = expectation.copy()

expectation_copy.id = None
another_suite.add_expectation(
    expectation=expectation_copy
)
Introduction to Data Quality with Great Expectations

Checking Expectations

print(
    expectation_copy in another_suite.expectations
)
True
Introduction to Data Quality with Great Expectations

Deleting Expectations

Add

.add_expectation()

suite.add_expectation(
    expectation=expectation
)

Delete

.delete_expectation()

suite.delete_expectation(
    expectation=expectation
)
Introduction to Data Quality with Great Expectations

Adjusting Expectations

Update the .value attribute and save changes:

expectation = gx.expectations.ExpectTableColumnCountToEqual(
    value=10
)

expectation.value = 11
expectation.save()

Ensure the Expectation belongs to a Suite, otherwise:

RuntimeError: Expectation must be added to ExpectationSuite before it can be saved.
Introduction to Data Quality with Great Expectations

Updating an Expectation Suite

suite = gx.ExpectationSuite(name="my_suite") 
validation_definition = gx.ValidationDefinition(
    data=batch_definition, suite=suite, name="my_validation_definition"
)
# Define Expectation
col_name_expectation = gx.expectations.ExpectColumnToExist(column="GHI")

# Add Expectation to Suite suite.add_expectation(expectation=col_name_expectation)
# Run Validation Definition associated with the Suite validation_results = validation_definition.run()
Introduction to Data Quality with Great Expectations

Saving an Expectation Suite

Save changes to the Suite before running the Validation Definition to avoid errors:

validation_results = validation_definition.run()
ResourceFreshnessAggregateError: ExpectationSuite 'my_suite' has changed since it 
has last been saved. Please update with `<SUITE_OBJECT>.save()`, then try your 
action again.
Introduction to Data Quality with Great Expectations

Saving an Expectation Suite

Use the .save() method to save the Suite and run the Validation Definition error-free:

suite.save()

validation_results = validation_definition.run()
print(validation_results.success)
False
Introduction to Data Quality with Great Expectations

Cheat sheet

Copy Expectation:

expectation_copy = expectation.copy()
expectation_copy.id = None

Check if Expectation is in Suite:

expectation in suite.expectations

Delete Expectation:

suite.delete_expectation(expectation)

Update Expectation value:

expectation.value = new_value

Save changes to Expectation:

expectation.save()

Save changes to Expectation Suite:

suite.save()
Introduction to Data Quality with Great Expectations

Let's practice!

Introduction to Data Quality with Great Expectations

Preparing Video For Download...