बेसिक कॉलम एक्सपेक्टेशंस

Great Expectations के साथ Data Quality परिचय

Davina Moossazadeh

Data Scientist

Shein Footwear डेटासेट

Kaggle के Shein Footwear डेटासेट की पहली पाँच पंक्तियाँ, pandas में लोडेड। DataFrame में ये कॉलम हैं: "name", "link", "price_usd", "mark_price_usd", "star_rating", "colour", "seller_name", "review_count", "sku_id", और "hero_image".

¹ https://www.kaggle.com/datasets/atharvataras/shein-footwear-dataset

Row-level Expectations

Row-Level Expectations हर पंक्ति पर अलग से लागू होती हैं
- तभी सफल होंगी जब शर्त हर पंक्ति के लिए सही हो

Row-level Expectations

Missingness Expectation

gx.expectations.ExpectColumnValuesToNotBeNull(
    column="colour"
)

Type Expectation

gx.expectations.ExpectColumnValuesToBeOfType(
    column="review_count", type_="str"
)

Aggregate-level Expectations: distinct values

Distinct values Expectation

gx.expectations.ExpectColumnDistinctValuesToEqualSet(
    column="seller_name", value_set={"Womens Shoes"}
)

Aggregate-level Expectations: unique value count

Unique value count Expectation

gx.expectations.ExpectColumnUniqueValueCountToBeBetween(
    column="review_count", min_value=5, max_value=101
)

Aggregate-level Expectations: uniqueness

Uniqueness Expectation

gx.expectations.ExpectColumnValuesToBeUnique(
    column="sku_id"
)

Aggregate-level Expectations: mode

Mode Expectation

gx.expectations.ExpectColumnMostCommonValueToBeInSet(
    column="colour", value_set={"Khaki", "Purple", "Grey"}
)

चीट शीट

Row-level Expectations:

ExpectColumnValuesToNotBeNull(
    column: str
)
ExpectColumnValuesToBeOfType(
    column: str, type_: str
)

Aggregate-level Expectations:

ExpectColumnDistinctValuesToEqualSet(
    column: str, value_set: set
)
ExpectColumnUniqueValueCountToBeBetween(
    column: str, 
    min_value: int, max_value: int
)
ExpectColumnValuesToBeUnique(column: str)
ExpectColumnMostCommonValueToBeInSet(
    column: str, value_set: set
)

अभ्यास करते हैं!

Great Expectations के साथ Data Quality परिचय