Finding patterns in missing data

Dealing with Missing Data in Python

Suraj Donthi

Deep Learning & Computer Vision Consultant

Finding correlations between missingness

• Missingness heatmap or correlation map
• Missingness dendrogram

Missingness Heatmap

• Graph of correlation of missing values between columns
• Explains the dependencies of missingness between columns

import missingno as msno
diabetes = pd.read_csv('pima-indians-diabetes data.csv')

msno.heatmap(diabetes)

Missingness Dendrogram

• Tree diagram of missingness
• Describes correlation of variables by grouping them

msno.dendrogram(diabetes)

Summary

• Analyze missingness heatmap

    msno.heatmap(df)
  

• Analayze missingness dendrogram

    msno.dendrogram(df)
  

Let's practice!

Dealing with Missing Data in Python