减少模型的特征

R 中的特征工程

Jorge Zazueta

Research Professor and Head of the Modeling Group at the School of Economics, UASLP

减少特征数量的原因

消除无关或低信息量变量的好处包括：

在不显著增加偏差的情况下降低方差
提升样本外表现
缩短计算时间
降低模型复杂度
提高可解释性

用变量重要性筛选特征

用全部特征拟合模型

lr_recipe_full <-
  recipe(Loan_Status ~., data = train) %>%
  update_role(Loan_ID, new_role = "ID")

lr_workflow_full <- 
  workflow() %>%
  add_model(lr_model) %>%
  add_recipe(lr_recipe_full)

lr_fit_full <- 
  lr_workflow_full %>%
  fit(data = train)

绘制变量重要性 vip 图

lr_fit_full %>%
  extract_fit_parsnip() %>%
  vip(aesthetics = list(fill = "steelblue"))

变量重要性 变量重要性的柱状图。

用公式语法构建精简模型

可直接使用基础 R 公式语法添加特征。

# 创建配方
recipe_formula <- 
  recipe(Loan_Status ~ Credit_History + Property_Area + 
           LoanAmount, data = train)

# 与模型打包
workflow_formula <- # 与模型打包
  workflow() %>% add_model(lr_model) %>%
  add_recipe(recipe_formula)

通过创建特征向量构建精简模型

可在训练前通过特征向量选择特征。

# 特征向量
features <- c("Credit_History", "Property_Area", "LoanAmount", "Loan_Status") 

# 训练与测试数据
train_features <- train %>% select(all_of(features))
test_features <- test %>% select(all_of(features))

# 创建配方并与模型打包
recipe_features <- recipe(Loan_Status ~., data = train_features)
workflow_features <- workflow() %>% add_model(lr_model) %>%
  add_recipe(recipe_features)

创建增强对象

两种方法的增强对象

lr_aug_formula <-
  workflow_formula %>%
  fit(data = train) %>%
  augment(new_data = test)

lr_aug_features <-
  workflow_features %>%
  fit(data = train_features) %>%
  augment(new_data = test_features)

两种方式返回相同结果

all_equal(lr_aug_features, 
lr_aug_formula %>%
select(all_of(features),
starts_with(".pred")))

[1] TRUE

比较完整模型与精简模型

使用全部特征

lr_fit_full <- # 训练工作流
  lr_workflow_full %>%
  fit(data = train)
lr_aug_full <- # 增强
  lr_fit_full %>%
  augment(test)
lr_aug_full %>% # 评估
  class_evaluate(truth = Loan_Status, 
                 estimate = .pred_class,
                 .pred_Y)

# A tibble: 2 × 3
  .metric  .estimator .estimate
  <chr>    <chr>          <dbl>
1 accuracy binary         0.842
2 roc_auc  binary         0.744

使用前三个特征*

lr_fit_formula <- # 训练工作流
  workflow_formula %>%
  fit(train)
lr_aug_formula <- # 增强
  lr_fit_formula %>%
  augment(new_data = test)
lr_aug_formula %>% # 评估
  class_evaluate(truth = Loan_Status, 
                 estimate = .pred_class,
                 .pred_Y)

# A tibble: 2 × 3
  .metric  .estimator .estimate
  <chr>    <chr>          <dbl>
1 accuracy binary         0.842
2 roc_auc  binary         0.733

Ayo berlatih!

R 中的特征工程