Math and Optimization
Content
Linear Algebra
- Vectors
- Norms & inner products
- Linear Independence
- Rotations
- Matrices
Unconstrained Optimization
- Convexity 101
- Gradient Descent
- Distributed Implementation
- Newton’s Method
- Broyden-Fletcher-Goldfarb-Shanno
Constrained Optimization
- Basic Convexity
- Linear & Quadratic Programs
- Interior Point Solvers
- Bundle Methods
Online Optimization
- Stochastic Gradient Descent
Discrete Problems
Supplementary material
- Nesterov and Vial (expected convergence) http://dl.acm.org/citation.cfm?id=1377347
- Bartlett, Hazan, Rakhlin (strong convexity SGD) https://proceedings.neurips.cc/paper/2007/file/261b909dfbee5a1a09d5eb50ed7a17e0-Paper.pdf
- TAO (toolkit for advanced optimization) http://www.mcs.anl.gov/research/projects/tao/
- Ratliff, Bagnell, Zinkevich http:**martin.zinkevich.orgpublicationsratliff_nathan_2007_3.pdf]
- Shalev-Shwartz, Srebro, Singer (Pegasos paper) http://dl.acm.org/citation.cfm?id=1273598
- Langford, Smola, Zinkevich (slow learners are fast) http://arxiv.org/abs/0911.0491
- Hogwild (Recht, Wright, Re) http://pages.cs.wisc.edu/~brecht/papers/hogwildTR.pdf
Videos
The content of chapters 1 and 2 was covered in the recitations.