Grid- und Random-Suche mit mlr

Hyperparameter-Tuning in R

Dr. Shirin Elsinghorst

Senior Data Scientist

Hyperparameter-Tuning mit mlr

In mlr musst du definieren:

den Suchraum für jeden Hyperparameter
die Tuning-Methode (z. B. Grid- oder Random-Suche)
die Resampling-Methode

Suchraum definieren

makeParamSet(
  makeNumericParam(),
  makeIntegerParam(),
  makeDiscreteParam(),
  makeLogicalParam(),
  makeDiscreteVectorParam()
)

                                       Type  len             Def
autoencoder                         logical    -           FALSE
use_all_factor_level                logical    -            TRUE 
activation                         discrete    -       Rectifier
hidden                        integervector <NA>         200,200 
epochs                              numeric    -              10
train_samples_per_iteration         numeric    -              -2 
seed                                integer    -               -

getParamSet("classif.h2o.deeplearning")

adaptive_rate                        logical    -            TRUE
rho                                  numeric    -            0.99
epsilon                              numeric    -           1e-08
rate                                 numeric    -           0.005

                                       Type  len             Def
autoencoder                         logical    -           FALSE
use_all_factor_level                logical    -            TRUE
activation                         discrete    -       Rectifier
hidden                        integervector <NA>         200,200
epochs                              numeric    -              10
train_samples_per_iteration         numeric    -              -2
seed                                integer    -               -
adaptive_rate                       logical    -            TRUE

rho                                 numeric    -            0.99
epsilon                             numeric    -           1e-08
rate                                numeric    -           0.005

param_set <- makeParamSet(
  makeDiscreteParam("hidden", values = list(one = 10, two = c(10, 5, 10))),
  makeDiscreteParam("activation", values = c("Rectifier", "Tanh")),
  makeNumericParam("l1", lower = 0.0001, upper = 1),
  makeNumericParam("l2", lower = 0.0001, upper = 1))

Tuning-Methode festlegen

Grid Search

ctrl_grid <- makeTuneControlGrid()
ctrl_grid

Tune control: TuneControlGrid
Same resampling instance: TRUE
Imputation value: <worst>
Start: <NULL>

Tune threshold: FALSE
Further arguments: resolution=10

Funktioniert nur mit diskreten Parametersätzen!

Random Search

ctrl_random <- makeTuneControlRandom()
ctrl_random

Tune control: TuneControlRandom
Same resampling instance: TRUE
Imputation value: <worst>
Start: <NULL>
Budget: 100
Tune threshold: FALSE
Further arguments: maxit=100

Resampling-Strategie definieren

cross_val <- makeResampleDesc("RepCV", 
                              predict = "both",
                              folds = 5 * 3)

param_set <- makeParamSet(...)
ctrl_grid <- makeTuneControlGrid()

task <- makeClassifTask(data = knowledge_train_data, 
                        target = "UNS")
lrn <- makeLearner("classif.h2o.deeplearning", 
                   predict.type = "prob", 
                   fix.factors.prediction = TRUE)

lrn_tune <- tuneParams(lrn,
                       task,
                       resampling = cross_val,
                       control = ctrl_grid,
                       par.set = param_set)

Hyperparameter tunen

lrn_tune <- tuneParams(lrn, task, resampling = cross_val, control = ctrl_grid, par.set = param_set)

[Tune-y] 27: mmce.test.mean=0.6200000; time: 0.0 min
[Tune-x] 28: hidden=two; activation=Rectifier; l1=0.578; l2=1
[Tune-y] 28: mmce.test.mean=0.6800000; time: 0.0 min
[Tune-x] 29: hidden=one; activation=Rectifier; l1=0.156; l2=0.68
[Tune-y] 29: mmce.test.mean=0.4400000; time: 0.0 min
[Tune-x] 30: hidden=one; activation=Rectifier; l1=0.717; l2=0.427
[Tune-y] 30: mmce.test.mean=0.6600000; time: 0.0 min

[Tune] Result: hidden=two; activation=Tanh; l1=0.113; 
l2=0.0973 : mmce.test.mean=0.2000000

# tictoc
26.13 sec elapsed

Lass uns üben!

Hyperparameter-Tuning in R