Random forests

Supervised Learning in R: Regression

Nina Zumel and John Mount

Win-Vector, LCC

Random Forests

Multiple diverse decision trees averaged together

Draw bootstrapped sample from training data
For each sample grow a tree
- At each node, pick best variable to split on (from a random subset of all variables)
- Continue until tree is grown
To score a datum, evaluate it with all the trees and average the results.

cnt ~ hr + holiday + workingday + 
  weathersit + temp + atemp + hum + windspeed

model <- ranger(fmla, bikesJan, 
                num.trees = 500, 
                respect.unordered.factors = "order")

formula, data
num.trees (default 500) - use at least 200
mtry - number of variables to try at each node
- default: square root of the total number of variables
respect.unordered.factors - recommend set to "order"
- "safe" hashing of categorical variables

model

Ranger result
...
OOB prediction error (MSE):       3103.623 
R squared (OOB):                  0.7837386

Random forest algorithm returns estimates of out-of-sample performance.

bikesFeb$pred <- predict(model, bikesFeb)$predictions

predict() inputs:

Predictions can be accessed in the element predictions.

Calculate RMSE:

bikesFeb %>% 
  mutate(residual = cnt - pred) %>%
  summarize(rmse = sqrt(mean(residual^2)))

      rmse
1 67.15169

Model	RMSE
Quasipoisson	69.3
Random forests	67.15

Supervised Learning in R: Regression