Introduction to Machine Learning

10-701/15-781, Carnegie Mellon University, Spring 2013

Practical Information

  • Lectures: Monday and Wednesday, 12:00PM to 1:20PM
  • Location: Baker Hall A51
  • Recitations: Tuesdays 5:00PM to 6:00PM
  • Location: Porter Hall 100 (January 22, 2013), Doherty Hall A302 (January 29, 2013 onwards)
  • Instructor: Barnabas Poczos (office hours 10am–12pm Thursdays in Gates 8231) and Alex Smola (office hours 2–4pm Tuesdays in Gates 8002)
  • TAs: Ina Fiterau (office hours 2–4pm Mondays in Gates 8021), Mu Li (office hours 5–6pm Fridays in Gates 7713), Junier Oliva (office hours 4:30–5:30pm Thursdays in Gates 8227), Xuezhi Wang (office hours 5–6pm Wednesdays in Gates 6503), Leila Wehbe (office hours 10:30–11:30am Wednesdays in Gates 8021)
  • Grading Policy: Homework (33%), Midterm (33%), Project (33%), Final (34%) with best 3 out of 4 used for score (final is mandatory)
  • Google Group: Join here for discussions and announcements

Overview

Machine learning studies the question “how can we build computer programs that automatically improve their performance through experience?” This includes learning to perform many types of tasks based on many types of experience. For example, it includes robots learning to better navigate based on experience gained by roaming their environments, medical decision aids that learn to predict which therapies work best for which diseases based on data mining of historical health records, and speech recognition systems that learn to better understand your speech based on experience listening to you.

This course is designed to give PhD students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The topics of the course draw from machine learning, classical statistics, data mining, Bayesian statistics and information theory. Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate. If you are interested in this topic, but are not a PhD student, or are a PhD student not specializing in machine learning, you might consider Roni Rosenfeld’s master’s level course on Machine Learning, 10-601.

Resources

For specific videos of the class go to the individual lectures. This is also where you will find pointers to further reading material etc.

Prerequisites

  • Basic probability and statistics are a plus.
  • Basic linear algebra (matrices, vectors, eigenvalues) is a plus. Knowing functional analysis would be great but not required.
  • Ability to write code that exceeds “Hello World.” Preferably beyond Matlab or R.
  • Basic knowledge of optimization. Having attended a convex optimization class would be great but the recitations will cover this.

Schedule

# Day Date Topic Lecturer
1 M January 14, 2013 Introduction to Machine Learning Alex
2 W January 16, 2013 Basic Statistics Barnabas
M January 21, 2013 Martin Luther King day
3 W January 23, 2013 Instance Based Learning Alex
4 M January 28, 2013 Perceptron Alex
5 W January 30, 2013 Support Vector Classification Alex
6 M February 4, 2013 Kernels Alex
7 W February 6, 2013 Kernels Alex
8 M February 11, 2013 Convergence Bounds Barnabas
9 W February 13, 2013 Risk Minimization Barnabas
10 M February 18, 2013 Learning Theory Barnabas
11 W February 20, 2013 Online Learning Barnabas
12 M February 25, 2013 Gaussian Processes Alex
13 W February 27, 2013 Exponential Families Alex
M March 4, 2013 Midterm exam Barnabas
14 W March 6, 2013 Principal Component Analysis Barnabas
M March 11, 2013 Spring break
W March 13, 2013 Spring break
15 M March 18, 2013 Directed Graphical Models Alex
16 W March 20, 2013 Dynamic Programming Alex
17 M March 25, 2013 Latent Variable Models Alex
18 W March 27, 2013 Sampling Alex
19 M April 1, 2013 Information Theory Barnabas
20 W April 3, 2013 Decision Trees Barnabas
21 M April 8, 2013 Neural Networks Barnabas
22 W April 10, 2013 Boosting Barnabas
23 M April 15, 2013 Kalman Filter Barnabas
24 W April 17, 2013 Reinforcement Learning Barnabas
25 M April 22, 2013 Scalability Alex
W April 24, 2013 Project Presentations Alex
M April 29, 2013 Project Presentations Alex
W May 1, 2013 Poster Session Alex
M May 6, 2013 Final Exam Barnabas

Lectures

  1. Introduction
  2. Basic Statistics
  3. Instance based methods
  4. Perceptron
  5. Support Vectors
  6. Kernels
  7. Kernels
  8. Stochastic Convergence
  9. Tail Bounds
  10. Risk Minimization
  11. Learning Theory
  12. Gaussian Processes
  13. Exponential Families
  14. PCA
  15. Graphical Models
  16. Dynamic Programming
  17. Latent Variables I
  18. Latent Variables II
  19. EM and Clustering
  20. ICA
  21. Sampling
  22. Trees
  23. Neural Networks
  24. Boosting
  25. Reinforcement Learning
  26. Scalability

Recitations

  1. Statistics
  2. Linear Algebra
  3. Optimization
  4. Model Selection
  5. Convergence Bounds
  6. Kernels
  7. Risk. Gaussian Processes
  8. Exponential Families
  9. Information Theory
  10. Bayesian Networks

Team