Optimisation avancée avec mlr

Optimisation des hyperparamètres en R

Dr. Shirin Elsinghorst

Data Scientist

Contrôles d’optimisation avancés

makeTuneControlCMAES : stratégie d’évolution CMA
makeTuneControlDesign : data frame prédéfini d’hyperparamètres
makeTuneControlGenSA : recuit simulé généralisé
makeTuneControlIrace : tuning par F-Racing itératif
makeTuneControlMBO : optimisation bayésienne (model-based)

# Recuit simulé généralisé
ctrl_gensa <- makeTuneControlGenSA()
# Créer un échantillonnage holdout
bootstrap <- makeResampleDesc("Bootstrap", predict = "both")

# Lancer l’optimisation
lrn_tune <- tuneParams(learner = lrn, task = task, 
                       resampling = bootstrap, 
                       control = ctrl_gensa, 
                       par.set = param_set,
                       measures = list(acc, mmce))

[Tune-x] 2170: eta=0.0771; max_depth=4
[Tune-y] 2170: acc.test.mean=0.9317275,mmce.test.mean=0.0682725; time: 0.0 m
[Tune-x] 2171: eta=0.822; max_depth=8
[Tune-y] 2171: acc.test.mean=0.9276912,mmce.test.mean=0.0723088; time: 0.0 m
[Tune-x] 2172: eta=0.498; max_depth=4
[Tune-y] 2172: acc.test.mean=0.9311626,mmce.test.mean=0.0688374; time: 0.0 m
[Tune-x] 2173: eta=0.365; max_depth=4
[Tune-y] 2173: acc.test.mean=0.9288406,mmce.test.mean=0.0711594; time: 0.0 m

# Créer un échantillonnage holdout
bootstrap <- makeResampleDesc("Bootstrap", predict = "both")
# Lancer l’optimisation
lrn_tune <- tuneParams(learner = lrn, 
                       task = task, 
                       resampling = bootstrap, 
                       control = ctrl_gensa, 
                       par.set = param_set,
                       measures = list(acc, setAggregation(acc, train.mean), 
                                       mmce, setAggregation(mmce, train.mean)))

[Tune-x] 3920: eta=0.294; max_depth=8
[Tune-y] 3920: acc.test.mean=0.9250118,
               acc.train.mean=0.9740000,
               mmce.test.mean=0.0749882,
               mmce.train.mean=0.0260000; 
               time: 0.0 min

Validation croisée imbriquée et rééchantillonnage imbriqué

lrn_wrapper <- makeTuneWrapper(learner = lrn, 
                               resampling = bootstrap, 
                               control = ctrl_gensa, 
                               par.set = param_set,
                               measures = list(acc, mmce))

Soit entraîner directement

model_nested <-  train(lrn_wrapper, task)
getTuneResult(model_nested)

Soit ajouter une validation croisée imbriquée en 2 passes

cv2 <- makeResampleDesc("CV", iters = 2)
res <- resample(lrn_wrapper, task, 
                resampling = cv2, 
                extract = getTuneResult)
generateHyperParsEffectData(res)

Choisir des hyperparamètres depuis un ensemble de tuning

lrn_best <- setHyperPars(lrn, par.vals = list(minsplit = 4, 
                                              minbucket = 3, 
                                              maxdepth = 6))

model_best <- train(lrn_best, task)

predict(model_best, newdata = knowledge_test_data)

Prediction: 30 observations
predict.type: response
threshold: 
time: 0.00
  truth response
1  High     High
2  High     High
3  High     High
4  High     High
5  High     High
6  High     High
... (#rows: 30, #cols: 2)

À vous de jouer !

Optimisation des hyperparamètres en R