Lasso 回归

R 中的降维

Matt Pickard

Owner, Pickard Predictives, LLC

Lasso 回归概览

监督式特征选择
L1 正则化
惩罚回归系数
收缩系数
次要系数收缩为 0
自然完成特征选择

linear_reg(engine = "glmnet", penalty = 0.001 , mixture = 1)

标准化数据

先标准化数据，使惩罚在各特征间等效
目标变量用 scale()
- 返回矩阵，需用 as.vector() 转为向量
预测变量用 step_normalize()

示例

# 缩放目标变量
df <- df %>% mutate(target = as.vector(scale(target))) 
... 
# 缩放预测变量
recipe() %>% step_normalize(all_numeric_predictors())

选择惩罚值

惩罚是需优化的超参数
搜索最佳惩罚值
在 tidymodels 中使用 tune()

linear_reg(engine = "glmnet", penalty = tune() , mixture = 1)

准备数据

缩放目标变量

house_sales_subset_df <- house_sales_subset_df %>% 
  mutate(price = as.vector(scale(price)))

创建训练集和测试集

split <- initial_split(house_sales_subset_df, prop = 0.8)
train <- split %>% training()
test <-  split %>% testing()

创建配方

lasso_recipe <- 
  recipe(price ~ ., data = train) %>% 
  step_normalize(all_numeric_predictors())

创建工作流

创建模型规范

lasso_model <- linear_reg(penalty = 0.01, mixture = 1, engine = "glmnet")

创建工作流

lasso_workflow <- workflow(preprocessor = lasso_recipe, spec =  lasso_model)

拟合工作流

tidy(lasso_workflow %>% fit(train)) %>% filter(estimate > 0)

# A tibble: 9 × 3
  term          estimate penalty
  <chr>            <dbl>   <dbl>
1 bathrooms      0.0477     0.01
2 sqft_living    0.434      0.01
3 floors         0.0262     0.01
4 waterfront     0.133      0.01
5 view           0.0510     0.01
6 condition      0.0319     0.01
...              ...        ...

创建可调的模型工作流

创建可调的模型规范

lasso_model <- linear_reg(penalty = tune(), mixture = 1, engine = "glmnet")
lasso_workflow <- workflow(preprocessor = lasso_recipe, spec =  lasso_model)

创建交叉验证训练样本

train_cv <- vfold_cv(train, v = 5)

创建惩罚值网格

penalty_grid <- grid_regular(penalty(range = c(-3, -1)), levels = 20)

将 0.001 到 0.1 的惩罚范围指定为 range = c(-3, -1)

拟合一组模型

创建拟合模型的网格

lasso_grid <- tune_grid(
  lasso_workflow,
  resamples = train_cv,
  grid = penalty_grid)

可视化模型表现

autoplot(lasso_grid, metric = "rmse")

惩罚值表现图

最终确定模型

获取最佳模型的惩罚值

best_rmse <- lasso_grid %>% select_best("rmse")

重新拟合最佳模型

final_lasso <- 
  finalize_workflow(lasso_workflow, best_rmse) %>% 
  fit(train)

显示最佳模型的系数

tidy(final_lasso) %>% filter(estimate > 0)

让我们练习！

R 中的降维