Tuning SVMs

Support Vector Machines in R

Kailash Awati

Instructor

Objective of tuning

Hard to find optimal values of parameters manually for complex kernels.
Objective: to find optimal set of parameters using tune.svm() function.

Tuning in a nutshell

How it works:
- set a range of search values for each parameter. Examples: cost = 10^(-1:3), gamma = c(0.1,1,10), coef0 = c(0.1,1,10)
- Build an SVM model for each possible combination of parameter values and evaluate accuracy.
- Return the parameter combination that yields the best accuracy.
Computationally intensive procedure!

Tune SVM model for the radially separable dataset created earlier
- Built polynomial kernel SVM in previous lesson
- Accuracy of SVM was ~94%.
Can we do better by tuning gamma, cost and coef0?

tune_out <- tune.svm(x = trainset[,-3], y = trainset[,3], 
                     type = "C-classification", kernel = "polynomial", degree = 2,
                     cost = 10^(-1:2), gamma = c(0.1,1,10), coef0 = c(0.1,1,10))

#print out tuned parameters
tune_out$best.parameters$cost
tune_out$best.parameters$gamma
tune_out$best.parameters$coef0

0.1
10
1

Build SVM model using best values of parameters from tune.svm().

svm_model <- svm(y ~ ., data = trainset, type = "C-classification", kernel = "polynomial", degree = 2,
        cost = tune_out$best.parameters$cost,
        gamma = tune_out$best.parameters$gamma,
        coef0 = tune_out$best.parameters$coef0)

evaluate training and test accuracy

pred_train <- predict(svm_model, trainset)
mean(pred_train == trainset$y)
pred_test <- predict(svm_model, testset)
mean(pred_test == testset$y)

1

0.9677419

#plot using svm plot
plot(svm_model, trainset)

Chapter 3.4 - radially separable dataset, tuned quadratic kernel, plot using svm.plot

Time to practice!

Support Vector Machines in R