Basic Tools

Introduction to Machine Learning - 10-701/15-781


  • Linear regression

    • Optimization problem

    • Examples

    • Overfitting

  • Parzen windows

    • Basic idea (smoothing over empirical average)

    • Kernels

  • Model selection

    • Overfitting and underfitting

    • Crossvalidation and leave-one-out estimation

    • Bias-variance tradeoff

    • Curse of dimensionality

  • Watson-Nadaraya estimators

    • Regression

    • Classification

  • Nearest neighbor estimator

    • Limit case via Parzen

    • Fast lookup


Annotated video is here.

Also see the first part of lecture 3.

Supplementary material

Slides available in PDF and Keynote. If you want to extract the equations from the slides you can do so by using LaTeXit, simply by dragging the equation images into it.