Optimization (Recitation)

Introduction to Machine Learning - 10-701/15-781

Content

  • Unconstrained problems

    • Gradient descent

    • Newton's method

    • Conjugate gradient descent

    • Broden-Fletcher-Goldfarb-Shanno (BFGS)

  • Convexity

    • Properties

    • Lagrange function

    • Wolfe dual

  • Batch methods

    • Distributed subgradient

    • Bundle methods

  • Online methods

    • Unconstrained subgradient

    • Gradient projections

    • Parallel optimization

Supplementary material

PDF slides in for Stochastic Gradient Descent and Quadratic Programing. If you want to extract the equations from the slides you can do so by using LaTeXit, simply by dragging the equation images into it.

Videos

Unedited video straight from a GF2.