Course on Machine Learning with Kernels

Hong Kong, ICONIP'06, October 3

Alex Smola
National ICT Australia, Machine Learning Program, Canberra Laboratory

Lecture 1: Exponential Families

We introduce exponential families and show how they can be used for modelling a large range of distributions important for supervised learning. In particular we will discuss multinomial and Gaussian families. Moreover, we show how optimization problems are solved in the case of normal priors. Finally, we discuss connections to graphical models and message passing. (Slides of Lecture 1)

Lecture 2: Conditional Models

By conditioning on location we extend exponential family models into state of the art multiclass classification and regression estimators. In addition, we will discuss conditional random fields, which are used for document annotation and named entity tagging. (Slides of Lecture 2)

Lecture 3: Maximum Mean Discrepancy

Operator methods are useful to test for identity between distributions. We will discuss a very simple and easily implementable criterion for such tests. Applications to data integration are discussed. We also discuss applications to covariate shift correction, that is, cases where training and test set are drawn from different distributions. (Slides of Lecture 3)

Lecture 4: Dependency Estimation

In a similar fashion to the two sample test above, we can also use operator methods for dependency tests. More specifically, we can use them to obtain contrast functions for independent component analysis and feature selection. We will discuss simple algorithms which achieve this goal. (Slides of Lecture 4)


Nothing beyond undergraduate knowledge in mathematics is expected. More specifically, I assume:

Relevant Links