Statistics

Slides in Keynote and PDF.

Content

• Probabilities

  • Bayes rule
  • Dependence, independence, conditional probabilities
  • Naive Bayes classifier
  • Probability estimation

• Tail bounds

  • Convergence of estimates
  • Chernoff, Hoeffding
  • Chebyshev
  • Gaussian
  • A/B testing and practical tail bounds

• Kernel density estimation

  • Parzen windows
  • Nearest neighbors
  • Model Selection
  • Watson-Nadaraya regression and classification

• Exponential families

  • Gaussian, multinomial and Poisson
  • Conjugate distributions and smoothing
  • Parameter estimation
  • Integrating out

Supplementary material

• Cover trees
• KD Trees
• Moore’s tutorial on KD trees and fast lookup structures
• Nadarya’s Regression Estimator
• Watson’s Classification Estimator
• Watson-Nadaraya regression package in R
• Stone’s k-NN regression consistency proof
• Cover and Hart’s k-NN classification consistency proof
• Tom Cover’s rate analysis for k-NN
Rates of Convergence for Nearest Neighbor Procedures.
• Sanjoy Dasgupta’s analysis for k-NN estimation with selective sampling
• Multiedit and Condense (Dasarathy, Sanchez, Townsend)
• Geometric approximation via core sets

Video