Regresi Lasso

Reduksi Dimensi di R

Matt Pickard

Owner, Pickard Predictives, LLC

Ikhtisar regresi Lasso

Seleksi fitur terawasi
Regularisasi L1
Memberi penalti pada koefisien regresi
Mengecilkan koefisien
Koefisien kurang penting menyusut ke nol
Secara alami melakukan seleksi fitur

linear_reg(engine = "glmnet", penalty = 0.001 , mixture = 1)

Standarkan data

Standarkan data dulu agar penalty setara di semua fitur
Gunakan scale() untuk variabel target
- mengembalikan matriks; ubah ke vektor dengan as.vector()
Gunakan step_normalize() untuk variabel prediktor

Contoh

# Skala variabel target
df <- df %>% mutate(target = as.vector(scale(target))) 
... 
# Skala variabel prediktor
recipe() %>% step_normalize(all_numeric_predictors())

Memilih nilai penalty

Penalty adalah hyperparameter yang dioptimalkan
Cari nilai penalty terbaik
Gunakan tune() di tidymodels

linear_reg(engine = "glmnet", penalty = tune() , mixture = 1)

Menyiapkan data

Skala variabel target

house_sales_subset_df <- house_sales_subset_df %>% 
  mutate(price = as.vector(scale(price)))

Buat set train dan test

split <- initial_split(house_sales_subset_df, prop = 0.8)
train <- split %>% training()
test <-  split %>% testing()

Buat recipe

lasso_recipe <- 
  recipe(price ~ ., data = train) %>% 
  step_normalize(all_numeric_predictors())

Buat workflow

Buat spesifikasi model

lasso_model <- linear_reg(penalty = 0.01, mixture = 1, engine = "glmnet")

Buat workflow

lasso_workflow <- workflow(preprocessor = lasso_recipe, spec =  lasso_model)

Fit workflow

tidy(lasso_workflow %>% fit(train)) %>% filter(estimate > 0)

# A tibble: 9 × 3
  term          estimate penalty
  <chr>            <dbl>   <dbl>
1 bathrooms      0.0477     0.01
2 sqft_living    0.434      0.01
3 floors         0.0262     0.01
4 waterfront     0.133      0.01
5 view           0.0510     0.01
6 condition      0.0319     0.01
...              ...        ...

Buat workflow model yang bisa dituning

Buat spesifikasi model yang bisa dituning

lasso_model <- linear_reg(penalty = tune(), mixture = 1, engine = "glmnet")
lasso_workflow <- workflow(preprocessor = lasso_recipe, spec =  lasso_model)

Buat sampel pelatihan cross-validation

train_cv <- vfold_cv(train, v = 5)

Buat grid nilai penalty

penalty_grid <- grid_regular(penalty(range = c(-3, -1)), levels = 20)

Rentang penalty 0.001 hingga 0.1 ditulis sebagai range = c(-3, -1)

Fit grid model

Buat grid model ter-fit

lasso_grid <- tune_grid(
  lasso_workflow,
  resamples = train_cv,
  grid = penalty_grid)

Plot kinerja model

autoplot(lasso_grid, metric = "rmse")

Plot kinerja penalty

plot kinerja penalty

Finalisasi model

Ambil nilai penalty untuk model terbaik

best_rmse <- lasso_grid %>% select_best("rmse")

Fit ulang model terbaik

final_lasso <- 
  finalize_workflow(lasso_workflow, best_rmse) %>% 
  fit(train)

Tampilkan koefisien model terbaik

tidy(final_lasso) %>% filter(estimate > 0)

Ayo berlatih!

Reduksi Dimensi di R