May 7, 2015


Overview of supervised learning

Predictive models that are calibrated on a set of training data and validated on a set of test data

What are Support Vector Machines?

SVM is typically used as a classification algorithm

  • It is an umbrella term for:
    • Maximal margin classifier
    • Support vector classifier
    • Support vector machine
  • It is a popular algorithm because:
    • It works well in a variety of settings
    • It sounds cool

Separating Hyperplanes

  • SVM classifies data with a separating hyperplane

  • A hyperplane is a flat surface with dimension p-1

  • If the data are linearly-seperable, there will be an infinite number of separating hyperplanes

Maximal Margin Classifier

  • The goal of the maximal margin classifier is to find a hyperplane with the widest margin between the two classes

  • We hope that a large margin on the training data will translate to a large margin on the test data

Support Vectors

  • Observations on the edge of the margin are called
    support vectors

  • If these points were moved slightly, the maximal margin hyperplane would move as well.