Decision-Tree for Regression

Machine Learning with Tree-Based Models in Python

Elie Kawerk

Data Scientist

Auto-mpg Dataset

auto

Machine Learning with Tree-Based Models in Python

Auto-mpg with one feature

auto1D

Machine Learning with Tree-Based Models in Python

Regression-Tree in scikit-learn

# Import DecisionTreeRegressor
from sklearn.tree import DecisionTreeRegressor
# Import train_test_split 
from sklearn.model_selection import train_test_split
# Import mean_squared_error as MSE
from sklearn.metrics import mean_squared_error as MSE

# Split data into 80% train and 20% test
X_train, X_test, y_train, y_test= train_test_split(X, y, 
                                                   test_size=0.2, 
                                                   random_state=3)

# Instantiate a DecisionTreeRegressor 'dt'
dt = DecisionTreeRegressor(max_depth=4, 
                           min_samples_leaf=0.1,
                           random_state=3)
Machine Learning with Tree-Based Models in Python

Regression-Tree in scikit-learn

# Fit 'dt' to the training-set
dt.fit(X_train, y_train)
# Predict test-set labels
y_pred = dt.predict(X_test)

# Compute test-set MSE
mse_dt =  MSE(y_test, y_pred)

# Compute test-set RMSE 
rmse_dt = mse_dt**(1/2)

# Print rmse_dt
print(rmse_dt) 

5.1023068889
Machine Learning with Tree-Based Models in Python

Information Criterion for Regression-Tree

impurity-reg

Machine Learning with Tree-Based Models in Python

Prediction

pred-reg

Machine Learning with Tree-Based Models in Python

Linear Regression vs. Regression-Tree

auto1D

Machine Learning with Tree-Based Models in Python

Let's practice!

Machine Learning with Tree-Based Models in Python

Preparing Video For Download...