SVM_Models.Rmd

---
title: Support Vector Machine Models
subtitle: ""
author: Andreas Handel
institute: "University of Georgia"
date: "`r file.mtime(knitr::current_input())`"
output: 
  html_document:
    toc_depth: 3
---


# Overview
In this unit, we will very briefly cover support vector machines (SVM), another useful type of machine learning/statistical model.

# Learning Objectives
* Be familiar with SVM.
* Understand advantages and disadvantages of SVM.


# Introduction

SVM were developed in the computer science and engineering communities and were among the best-performing types of models for a while. They are not that exceptional anymore (now the big trend is neural networks), but still often perform rather well, especially sometimes for data where other models don't perform quite as well. So SVM are often a good idea to try.


# General Idea behind SVM

SVM have cost/objective functions that do not penalize discrepancy between true and predicted outcome if the two are within a certain range, and do penalize outside that range. This is different than penalties based on basic least squares, which implement a penalty for any difference between model and data. SVM divide the predictor space into areas using multi-dimensional hyperplanes until observations within the same "volume" are sufficiently accurately predicted (a bit like trees).

SVM can come in different flavors, for instance depending on how discrepancies between data and model are penalized (absolute distance, quadratic, etc.) or how the boundaries of the hyperplanes are defined.


# Strengths and Weaknesses of SVM

SVM can be used for both classification and regression. They are very flexible and often have very good predictive performance. They are often well suited for data that contain outliers. 

The math behind SVM is not that easy to follow, and since SVM models are somewhat complex, they tend to be a bit of a _black box_ and the model itself is hard to interpret. 



# SVM in `tidymodels`

Tidymodels/parsnip [has several SVM models](https://parsnip.tidymodels.org/reference/index.html), such as `svm_linear()` and `svm_poly()`. They can be used as part of the `tidymodels` workflow like any other ML model you have already seen. They also need tuning. 



# Further Resources

To learn a bit more about SVM, you can check out [chapter 9 of ISL](https://statlearning.com/) which discusses SVM. So does [chapter 14 of HMLR](https://bradleyboehmke.github.io/HOML/svm.html).