Linear Methods

Slides in Keynote and PDF.

Content

• Perceptron

  • Neurons and Learning

  • Perceptron learning rule

  • Convergence Analysis

• Nonlinearity and Kernels

  • Preprocessing

  • Perceptron on features

  • Kernels and Kernel Perceptron

• Support Vector machines

  • Large Margin Classification

  • Soft Margin Loss

  • Kernel Trick

  • Regression

  • Novelty Detection

• Efficient Implementation

  • Optimization and active sets

  • Hash kernels for high-dimensional data

Supplementary material

• F. Rosenblatt, The perceptron: A probabilistic model for information storage and organization in the brain, Psychological Review, Vol 65(6), Nov 1958, 386-408.

• Y. Freund and R. Schapire, Large margin classification using the perceptron algorithm, In Machine Learning 37(3):277-296, 1999.

• N. Aronszajn, Theory of reproducing Kernels, Trans. Amer. Math. Soc. 68 (1950), 337-404

• G. Wahba, Spline Models for Observational Data, SIAM 1990

• B. Scholkopf, A. Smola, K. Muller, Nonlinear Component Analysis as Kernel Eigenvalue Problem, Neural Computation, 10:1299-1319, 1998

• S. Mika et al, Kernel-PCA and Denoising in Feature Spaces, NIPS 1999

• B. Scholkopf et al, Support Vector Method for Novelty Detection, Neural Computation, 2000

• A. Smola, L. Song, C.H. Teo, Relative Novelty Detection, NIPS 2009

• B. Scholkopf et al., Shrinking the Tube: A New Support Vector Regression Algorithm, Neural Computation, 1999

Videos