Large Scale Modeling

Content

This lecture covers latent variable model templates, structured estimation, and large scale inference in graphical models.

Latent Variable Model Templates

• Sets

  • Clusters
  • Trees (and hierarchical clustering)
  • Topics (and mixtures)
  • Supervised variants

• Chains

  • Markov models
  • Simplical mixtures
  • Semi-Markov models

• Factorizations

  • Latent Factor models
  • Hierarchical topic models
  • Recommender systems revisited

Structured Estimation

• Conditional Random Fields

  • Sequence annotation
  • Tagging
  • Semi-Markov models / sequence segmentation

• Max-Margin Estimation

  • Large Margin formulation
  • Scaled Margin variants
  • Approximate margin
  • Dual variants

• Applications

  • Segmentation
  • Annotation (gene finding, entity tagging)
  • F1/AUC score optimization
  • Ranking
  • Robust and invariant estimation

Large Scale Inference in Graphical Models

• Variational methods

  • Global variable models (clustering, topics)
  • Variational iterations
  • Bulk synchronous algorithms

• Message passing

  • Parallel updates
  • Graph coloring
  • Parallel Gibbs Sampler
  • Graphlab

• Approximations for Scalable Inference

  • Large local state
  • Large global state
  • Out of core storage
  • Asynchronous scheduling
  • Fast sampling (item ordering, heaps, fast proposals, SIMD)

Supplementary material

Slides in PDF and Keynote. If you want to extract the equations from the slides you can do so by using LaTeXit, simply by dragging the equation images into it.