The interrogation phase: data quality

AI for Data Analysts

Andy Cotgreave

Author and Speaker, Former Technical Evangelist at Tableau

The usual suspects

data_quality_issues1.jpg

AI for Data Analysts

The harder problems AI unlocks

data_quality_issues2.jpg

AI for Data Analysts

Fuzzy duplicates

 

  • Traditional: hand-craft rules, maintain forever
  • AI: writes the matching logic and brings semantic understanding

 

  • Output → a query or grouping you can read and adjust

fuzzy_duplicates.png

AI for Data Analysts

Time-order violations

time_order.jpg

  • Traditional: write SQL that compares timestamps
  • AI: describe what shouldn't happen → AI writes the query
AI for Data Analysts

Logical inconsistencies

logical_inconsistencies.jpg

 

  • Traditional: reference tables — someone has to maintain them
  • AI: world knowledge, no reference data required
AI for Data Analysts

Interrogate the AI's work

 

  • AI isn't magic — it's generating queries, code, or rules
  • That work can be wrong: bad regex, hallucinated business rule

ai_interrogation.jpg

AI for Data Analysts

Fixing the issues

clean_data1.jpg

clean_data2.jpg

AI for Data Analysts

Let's practice!

AI for Data Analysts

Preparing Video For Download...