Statistics

Slides in Keynote and PDF.

Content

• Probabilities

  • Bayes rule

  • Dependence, independence, conditional probabilities

  • Naive Bayes classifier

  • Probability estimation

• Tail bounds

  • Convergence of estimates

  • Chernoff, Hoeffding

  • Chebyshev

  • Gaussian

  • A/B testing and practical tail bounds

• Kernel density estimation

  • Parzen windows

  • Nearest neighbors

  • Model Selection

  • Watson-Nadaraya regression and classification

• Exponential families

  • Gaussian, multinomial and Poisson

  • Conjugate distributions and smoothing

  • Parameter estimation

  • Integrating out

Supplementary material

• Cover trees

• KD Trees

• Moore’s tutorial on KD trees and fast lookup structures

• Nadarya's Regression Estimator

• Watson's Classification Estimator

• Watson-Nadaraya regression package in R


• Stone’s k-NN regression consistency proof

• Cover and Hart’s k-NN classification consistency proof


• Tom Cover’s rate analysis for k-NN
Rates of Convergence for Nearest Neighbor Procedures.

• Sanjoy Dasgupta’s analysis for k-NN estimation with selective sampling


• Multiedit and Condense (Dasarathy, Sanchez, Townsend)


• Geometric approximation via core sets


Video