移除離群值

Feature Engineering for Machine Learning in Python

Robert O'Callaghan

Director of Data Science, Ordergroove

什麼是離群值？

分佈示意圖

Feature Engineering for Machine Learning in Python

以分位數偵測

Feature Engineering for Machine Learning in Python

在 Python 中計算分位數

q_cutoff = df['col_name'].quantile(0.95)

mask = df['col_name'] < q_cutoff

trimmed_df = df[mask]
Feature Engineering for Machine Learning in Python

以標準差偵測

Feature Engineering for Machine Learning in Python

在 Python 中以標準差偵測

mean = df['col_name'].mean()
std = df['col_name'].std()

cut_off = std * 3

lower, upper = mean - cut_off, mean + cut_off

new_df = df[(df['col_name'] < upper) & 
                 (df['col_name'] > lower)]
Feature Engineering for Machine Learning in Python

一起來練習吧！

Feature Engineering for Machine Learning in Python

Preparing Video For Download...