SML: Optimization

STATISTICS 241B, COMPUTER SCIENCE 281B

Content

  • Unconstrained problems

    • Gradient descent

    • Newton's method

    • Conjugate gradient descent

    • Broden-Fletcher-Goldfarb-Shanno (BFGS)

  • Convexity

    • Properties

    • Lagrange function

    • Wolfe dual

  • Batch methods

    • Distributed subgradient

    • Bundle methods

  • Online methods

    • Unconstrained subgradient

    • Gradient projections

    • Parallel optimization

Supplementary material

Slides in PDF and Keynote. If you want to extract the equations from the slides you can do so by using LaTeXit, simply by dragging the equation images into it. There's also an optimization chapter from the Learning with Kernels book.

Videos

This is unedited video straight from a Lumix GF2 with a 20mm lens which should explain the sound (it doesn't have a dedicated audio input) … But it should help as a supplement with the slides (YouTube typically makes the 1080i version available within 1 week of the upload).