Introduction to Machine Learning

10-701/15-781

Practical information

Office Hours

Monday 1-3pm Alex Smola Gates Hillman 8002
Tuesday 10-11am Leila Wehbe Gates Hillman 8021
Wednesday 1-3pm Geoff Gordon Gates Hillman 8105
Wednesday 5-6pm Ahmed Hefny Gates Hillman 8223
Thursday 10-11am Jing Xiang Gates Hillman 8009
Thursday 4-5pm Carlton Downey Gates Hillman 8007
Friday 2:30-3:30pm Dougal Sutherland Gates Hillman 6505

Updates

  • 090913 Initial site update

  • 091713 Moved site from ixwebhosting to a dedicated host on Amazon EC2. Performance should be much better now.

  • 093013 Content updates

Overview

Machine learning studies the question “how can we build computer programs that automatically improve their performance through experience?” This includes learning to perform many types of tasks based on many types of experience. For example, it includes robots learning to better navigate based on experience gained by roaming their environments, medical decision aids that learn to predict which therapies work best for which diseases based on data mining of historical health records, and speech recognition systems that learn to better understand your speech based on experience listening to you.

This course is designed to give PhD students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The topics of the course draw from machine learning, classical statistics, data mining, Bayesian statistics and information theory. Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.

Resources

For specific videos of the class, as well as slides, go to the individual lectures in the schedule below or the menu at left. This is also where you'll find pointers to further reading material etc.

Prerequisites

  • Basic probability and statistics are a plus.

  • Basic linear algebra (matrices, vectors, eigenvalues) is a plus. Knowing functional analysis would be great but not required.

  • Ability to write code that exceeds 'Hello World’. Preferably beyond Matlab or R.

  • Basic knowledge of optimization. Having attended a convex optimization class would be great but the recitations will cover this.

  • You should have no trouble answering the questions of the self evaluation handed out for the 10-601 course.

Schedule

Date Topic Lecturer
1 M September 9 Introduction to Machine Learning Alex
2 W September 11 Basic Tools and Density Estimation Alex
3 M September 16 Density Estimation and Basic Probability Alex + Geoff
R1 T September 17 Linear algebra review Jing
4 W September 18 Naive Bayes Geoff
5 M September 23 Perceptron Geoff
R2 T September 24 Probability review Dougal
6 W September 25 Perceptron + Kernels Alex
7 M September 30 Optimization Alex
R3 T October 1 Kernels, convexity review Leila
8 W October 2 Optimization 2 Alex
F October 4 HW1 due at noon (extended)
9 M October 7 Projects, story so far, Lagrange multipliers Geoff
R4 T October 8 Optimization review Dougal
10 W October 9 Duality Geoff
F October 11 Project proposal due at noon
11 M October 14 Duality & SVM Alex + Geoff
R5 T October 15 Duality and SVM Ahmed
12 W October 16 Kernel Methods Alex
M October 21 HW2 due at 10:30am (code handout, convexity notes)
13 M October 21 Kernel Methods Alex
R6 T October 22 Midterm Practice Jing
14 W October 23 Novelty Detection, Regularization and nu-Trick Alex
M October 28 Midterm exam (Midterm Practice, Solutions)
R7 T October 29 Tail Bounds & Averages Ahmed
15 W October 30 Tail Bounds & Averages Alex
16 M November 4 Tail Bounds & Averages Alex
R8 T November 5 Learning Theory Leila
17 W November 6 Learning Theory Alex
18 M November 11 Bootstrap Geoff
R9 T November 12 Information Theory Carlton
19 W November 13 Graphical Models: Bayes nets, dynamic programming on graphs Geoff
W November 13 HW3 due at 10:30am (code handout)
20 W November 18 Graphical Models: factor graphs, Markov random fields, junction trees Geoff
R10 T November 19 Graphical Models review Dougal
21 W November 20 Graphical Models: junction trees, belief propagation Geoff
22 M November 25
T November 26 no recitation (Thanksgiving)
W November 27 no class (Thanksgiving)
W November 27 HW4 due at 11:59pm (handout)
W November 27 Due date to sign up for extra credit assignment
23 M December 2
T December 3 Poster session, 3-6pm NSH Atrium
24 W December 4
W December 11 Project final report due at 11:59pm
Th December 12 Extra credit due at 11:59pm