Risk Minimization and Tail Bounds

Content

• Risk and loss

  • Loss functions
  • Risk
  • Empirical risk vs True risk
  • Empirical Risk minimization

• Underfitting and Overfitting

• Examples

  • Classification
  • Regression

Supplementary material

• Slides in PDF.
• L. Devroye, L. Gyorfi, G. Lugosi: A Probabilistic Theory of Pattern Recognition. Springer, New York, 1996.
• L. Gyorfi (editor): Principles of Nonparametric Learning, Springer-Verlag Wien New York, 2002
• L. Gyorfi, M. Kohler, A. Krzyzak, H. Walk: A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York, 2002
• A. Tsybakov: Introduction to Nonparametric Estimation, Springer Series in Statistics, 2008
• Robert D. Nowak: Lecture notes about Statistical Learning Theory

Videos

See the YouTube playlist for available lecture videos.