降低模型特徵數

R 的特徵工程

Jorge Zazueta

Research Professor and Head of the Modeling Group at the School of Economics, UASLP

為何要減少特徵數量

移除無關或資訊量低的變數有多種好處，包括：

降低模型變異而不大幅增加偏差
提升樣本外效能
縮短運算時間
降低模型複雜度
提高可解釋性

用變數重要性來篩選資料

使用所有特徵來配適模型

lr_recipe_full <-
  recipe(Loan_Status ~., data = train) %>%
  update_role(Loan_ID, new_role = "ID")

lr_workflow_full <- 
  workflow() %>%
  add_model(lr_model) %>%
  add_recipe(lr_recipe_full)

lr_fit_full <- 
  lr_workflow_full %>%
  fit(data = train)

繪製變數 vip 圖

lr_fit_full %>%
  extract_fit_parsnip() %>%
  vip(aesthetics = list(fill = "steelblue"))

變數重要性 變數重要性的長條圖。

用公式語法建立精簡模型

可以用基本的 R 公式語法直接選特徵。

# Create recipe
recipe_formula <- 
  recipe(Loan_Status ~ Credit_History + Property_Area + 
           LoanAmount, data = train)

# Bundle with model
workflow_formula <- # Bundle with model
  workflow() %>% add_model(lr_model) %>%
  add_recipe(recipe_formula)

用特徵向量建立精簡模型

也可以先建立特徵向量，再據此在訓練前篩選特徵。

# Feature vector
features <- c("Credit_History", "Property_Area", "LoanAmount", "Loan_Status") 

# Training and testing data
train_features <- train %>% select(all_of(features))
test_features <- test %>% select(all_of(features))

# Create recipe and bundle with model
recipe_features <- recipe(Loan_Status ~., data = train_features)
workflow_features <- workflow() %>% add_model(lr_model) %>%
  add_recipe(recipe_features)

建立增廣物件（augmented objects）

兩種方法的增廣物件

lr_aug_formula <-
  workflow_formula %>%
  fit(data = train) %>%
  augment(new_data = test)

lr_aug_features <-
  workflow_features %>%
  fit(data = train_features) %>%
  augment(new_data = test_features)

兩種作法回傳相同結果

all_equal(lr_aug_features, 
lr_aug_formula %>%
select(all_of(features),
starts_with(".pred")))

[1] TRUE

比較完整模型與精簡模型

使用所有特徵

lr_fit_full <- # Fit workflow
  lr_workflow_full %>%
  fit(data = train)
lr_aug_full <- # Augment
  lr_fit_full %>%
  augment(test)
lr_aug_full %>% # Evaluate
  class_evaluate(truth = Loan_Status, 
                 estimate = .pred_class,
                 .pred_Y)

# A tibble: 2 × 3
  .metric  .estimator .estimate
  <chr>    <chr>          <dbl>
1 accuracy binary         0.842
2 roc_auc  binary         0.744

使用前 3 大特徵*

lr_fit_formula <- # Fit workflow
  workflow_formula %>%
  fit(train)
lr_aug_formula <- # Augment
  lr_fit_formula %>%
  augment(new_data = test)
lr_aug_formula %>% # Evaluate
  class_evaluate(truth = Loan_Status, 
                 estimate = .pred_class,
                 .pred_Y)

# A tibble: 2 × 3
  .metric  .estimator .estimate
  <chr>    <chr>          <dbl>
1 accuracy binary         0.842
2 roc_auc  binary         0.733

一起來練習吧！

R 的特徵工程