Final Thoughts

Feature Engineering with PySpark

John Hogue

Lead Data Scientist

What you learned!

• Inspecting visually & statistically
• Dropping rows and columns
• Scaling and adjusting data
• Handling missing values
• Joining external datasets

• Generating features
• Extracting variables from messy fields
• Binning, bucketing and encoding
• Training and evaluating a model
• Interpreting model results

Time to learn something new!

Feature Engineering with PySpark