Introduction to Machine Learning - 10-701/15-781


Machine learning studies the question “how can we build computer programs that automatically improve their performance through experience?” This includes learning to perform many types of tasks based on many types of experience. For example, it includes robots learning to better navigate based on experience gained by roaming their environments, medical decision aids that learn to predict which therapies work best for which diseases based on data mining of historical health records, and speech recognition systems that learn to better understand your speech based on experience listening to you.

This course is designed to give PhD students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The topics of the course draw from machine learning, classical statistics, data mining, Bayesian statistics and information theory. Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate. If you are interested in this topic, but are not a PhD student, or are a PhD student not specializing in machine learning, you might consider Roni Rosenfeld's master's level course on Machine Learning, 10-601.


1. Introduction to Machine Learning

2. Basic Statistics

3. Instance Based Learning

4. Perceptron

5. Support Vector Classification

6. Kernels

7. Kernels

8. Stochastic Convergence

9. Tail bounds

10. Risk Minimization

11. Learning Theory

12. Gaussian Processes

13. Exponential Families

14. Principal Component Analysis

15. Directed Graphical Models

16. Dynamic Programming

17. Latent Variable Models

18. Sampling

19. Information Theory

20. Decision Trees

21. Neural Networks

22. Boosting

23. Kalman Filter

24. Reinforcement Learning

25. Scalability

Final Project Presentations

Each team gets to present a poster. Good teams get to present a spotlight and the six best teams projects get talks. Make sure you discuss what you're doing, why you're doing it, in which way it is different or better than what's available, and what it is good for.