處理遺漏值（I）

Feature Engineering for Machine Learning in Python

Robert O'Callaghan

Director of Data Science, Ordergroove

整列刪除（Listwise deletion）

      SurveyDate      ConvertedSalary     Hobby ... \
0  2/28/18 20:20                  NaN       Yes ...
1  6/28/18 13:26              70841.0       Yes ...
2    6/6/18 3:37                  NaN        No ...
3    5/9/18 1:06              21426.0       Yes ...
4  4/12/18 22:41              41671.0       Yes ...
Feature Engineering for Machine Learning in Python

在 Python 中進行整列刪除

# 刪除所有含至少一個遺漏值的列
df.dropna(how='any')
Feature Engineering for Machine Learning in Python

在 Python 中進行整列刪除

# 只刪除指定欄位有遺漏值的列
df.dropna(subset=['VersionControl'])
Feature Engineering for Machine Learning in Python

刪除法的問題

  • 會刪掉有效資料點
  • 假設遺漏是隨機的
  • 降低資訊量
Feature Engineering for Machine Learning in Python

以字串替換

# 將指定欄位的遺漏值
# 替換為給定字串
df['VersionControl'].fillna(
    value='None Given', inplace=True
)
Feature Engineering for Machine Learning in Python

記錄遺漏狀態

# 紀錄哪些值不是遺漏
df['SalaryGiven'] = df['ConvertedSalary'].notnull()

# 刪除指定欄位
df.drop(columns=['ConvertedSalary'])
Feature Engineering for Machine Learning in Python

練習時間

Feature Engineering for Machine Learning in Python

Preparing Video For Download...