Kompromis mezi biasem a variancí

Machine Learning with Tree-Based Models in R

Sandro Raabe

Data Scientist

Hyperparametry

  • Volí modelář
  • např. tree_depth
  • Zkontrolujte dokumentaci!
?decision_tree

několik hyperparametrů

Machine Learning with Tree-Based Models in R

Jednoduchý model

simple_spec <- decision_tree(tree_depth = 2) %>% 
    set_mode("regression")

simple_spec %>% fit(final_grade ~ .,
                    data = training_data)

Složitý model

complex_spec <- decision_tree(tree_depth = 15) %>% 
    set_mode("regression")

complex_spec %>% fit(final_grade ~ .,
                     data = training_data)

strom hloubky 2

strom hloubky 30

Machine Learning with Tree-Based Models in R

Složitý model – overfitting – vysoká variance

Predikce na trénovací sadě: výborně!

malé chyby na trénovací sadě

mae(train_results, 
    estimate = .pred,
    truth = final_grade)

# A tibble: 1 x 3
  .metric  .estimate
1 mae          0.204

Predikce na testovací sadě: zdaleka ne!

velké chyby na testovací sadě

mae(test_results, 
    estimate = .pred,
    truth = final_grade)

# A tibble: 1 x 3
  .metric  .estimate
1 mae          0.947
Machine Learning with Tree-Based Models in R

Jednoduchý model – underfitting – vysoký bias

Velké chyby na trénovací i testovací sadě:

bind_rows(training = mae(train_results, estimate = .pred, truth = final_grade),
          test     = mae(test_results,  estimate = .pred, truth = final_grade),
          .id = "dataset")

# A tibble: 2 x 4
  dataset    .metric  .estimate
  <chr>      <chr>        <dbl>
1 training   mae          0.754
2 test       mae          0.844
Machine Learning with Tree-Based Models in R

Kompromis mezi biasem a variancí

kompromis mezi biasem a variancí

 

  • Jednoduché modely -> vysoký bias
  • Složité modely -> vysoká variance
  • Kompromis mezi biasem a variancí
  • Modely stavět kolem optimálního bodu
Machine Learning with Tree-Based Models in R

Detekce overfittingu

Mimo vzorek/CV:

collect_metrics(cv_fits)



# A tibble: 1 x 3
  .metric    mean     n 
1 mae       2.432     5
  • Vysoká CV chyba
  • Overfitting / vysoká variance
  • Snižte složitost!

Na trénovacích datech:

mae(training_pred, 
    estimate = .pred, 
    truth = final_grade)

# A tibble: 1 x 2
  .metric  .estimate
1 mae          0.228
  • Malá trénovací chyba
Machine Learning with Tree-Based Models in R

Detekce underfittingu

Na trénovacích datech:

mae(training_pred, estimate = .pred, truth = final_grade)

# A tibble: 1 x 2
  .metric .estimate
  <chr>       <dbl>
1 mae         2.432
  • Velká chyba na trénovacích datech
  • Underfitting / vysoký bias
  • Zvyšte složitost!
Machine Learning with Tree-Based Models in R

Pojďme na to!

Machine Learning with Tree-Based Models in R

Preparing Video For Download...