Visualizing linear SVMs

Support Vector Machines in R

Kailash Awati

Instructor

Plot the training data using ggplot().

# Visualize training data, distinguish classes using color
p <- ggplot(data = trainset, aes(x = x1, y = x2, color = y)) +
     geom_point() +
     scale_color_manual(values = c("red", "blue"))
# Render plot
p

Mark out the support vectors using index from svm_model.

# Identify support vectors
df_sv <- trainset[svm_model$index, ]
# Mark out support vectors in plot
p <- p + geom_point(data = df_sv,
                    aes(x = x1, y = x2),
                    color = "purple",
                    size = 4, alpha = 0.5)
# Display plot
p

Chapter 2.2 - linearly separable data, default cost linear kernel with support vectors

Find slope and intercept of the boundary:

Build the weight vector, w, from coefs and SV elements of svm_model.

# Build weight vector
w <- t(svm_model$coefs) %*% svm_model$SV

slope =-w[1] / w[2]

# Calculate slope and save it to a variable
slope_1 <- -w[1] / w[2]

intercept = svm_model$rho / w[2]

# Calculate intercept and save it to a variable
intercept_1 <- svm_model$rho / w[2]

Add decision boundary using slope and intercept calculated in previous slide.
We use geom_abline() to add the decision boundary to the plot.

# Plot decision boundary based on calculated slope and intercept
p <- p + geom_abline(slope = slope_1,
                     intercept = intercept_1)

Margins parallel to decision boundary, offset by 1 / w[2] on either side of it.

# Add margins to plot
p <- p + 
    geom_abline(slope = slope_1,
                intercept = intercept_1 - 1 / w[2],
                linetype = "dashed") + 
    geom_abline(slope = slope_1,
                intercept = intercept_1 + 1 / w[2],
                linetype = "dashed")
# Display plot
p

Chapter 2.2 - linearly separable data, default cost linear kernel with support vectors, decision and margin boundaries

Soft margin classifiers

Allow for uncertainty in location / shape of boundary
- Never perfectly linear
- Usually unknown
Our decision boundary is linear, so we can reduce margin

Visualizing the decision boundary using the svm plot() function

The svm plot() function in e1071 offers an easy way to plot the decision boundary.

# Visualize decision boundary using built in plot function
plot(x = svm_model,
     data = trainset)

Chapter 2.2 - linearly separable dataset, default cost linear kernel, plot using svm.plot

Time to practice!

Support Vector Machines in R