# Machine Learning with Exponential Families

### Overview

Machine learning is an exciting new subject dealing with automatic recognition
of patterns (e.g. automatic recognition of faces, postal codes on envelopes,
speech recognition), prediction (e.g. predicting the stock market), datamining
(e.g. finding good customers, fraudulent transactions), applications in
bioinformatics (finding relevant genes, annotating the genome, processing DNA
Microarray images), or internet-related problems (spam filtering, searching,
sorting, network security). It is becoming a key area for technical advance and
people skilled in this area are worldwide in high demand.
### Formal Description

This unit introduces the fundamentals of machine learning,
based on the unifying framework of exponential families. The course
requires mathematical and computer skills. It will cover linear algebra and numerical
mathematics techniques, linear classification, regression, mathematical
programming, the Perceptron, Graphical Models, Boosting, density
estimation, conditional random fields, Regression methods, Kernels and
Regularization.
### Presenters

Prof. Stephane Canu

Dr. Alexander J. Smola

Dr. Vishy Vishwanathan

RSISE, Australian National University

Canberra, ACT 0200

Phone : (+61) 2 6125-8652

Cellular: (+61) 410 457 686

Fax: (+61) 2 6125-8651

[email protected]

### Dates of the Course

The course will be held on **May 4 - July 2**, with lectures every
Tuesday and Thursday at **2pm-4pm** in the **video conferencing
facility of the Crisp lecture theater**. Completion of the
assignment sheets is optional, however I **strongly encourage students** too
solve the problems.

The assessment will be in the form of oral exams, probably one week after the
end of the lectures.

### Prerequisites

Fourier Transforms
Linear Algebra (eigenvalues, matrix inverses, etc.)
programming skills if you want to apply the methods
The first week will be a quick refresher course in linear algebra and
statistics.
### Contents

Lectures

**Week 1:** Introduction and Mathematical Basics
( Lecture 1 and Lecture 2 )
**Week 2:** Density Estimation and Exponential Families
( Lecture 1 and Lecture 2 )
**Week 3:** Kernels ( Lecture 1 and 2 )
**Week 4:** Classification and Regression ( Lecture 1 and 2 )
**Week 5:** Optimization ( Lecture 1 and 2 )
**Week 6:** Graphical Models
**Week 7:** Graphical Models
**Week 8:** Conditional Random Fields
**Week 9:** Boosting
Assignments

**Assignment 1:** Introduction to conditional probabilities
(Assignment 1)
**Assignment 2:** Classical pattern recognition algorithms
(Assignment 2)
**Assignment 3:** Prior knowledge
(Assignment 3)
Scribe Notes

Each student is expected to scribe one or two lectures of vishy
The LaTeX template to use is here
Use this file for definitions Definitions.tex. Unless you really need it do *not* define new definitions. Please discuss with vishy before making changes to this file!
Scribe notes are due 2 weeks after the lecture and are a part of your assesment.
### Software

MATLAB. Here's a
quick
start guide.
SVLab Toolbox
GCC. See Brian Kernigham's
tutorial or a
list
of other C tutorials.

Last modified May 17, 2004