TEACHING / SPRING 2026

Optimization for Data Science.

16:960:690:01 / 16:954:694:02 · Rutgers University · Spring 2026

An in-depth graduate course on classical and modern optimization methods for statistics, machine learning, and data science.

Course Overview

The course begins with optimization models for learning from data and the theory of convex optimization. It then studies stochastic gradient methods, modern nonconvex optimization, and optimizers used in deep learning and reinforcement learning. Applications include statistical modeling, causal learning, robust optimization, and generative systems.

Emphasis is placed on modeling intuition, algorithm design, mathematical analysis, performance guarantees, and the limits of each method. Students learn to choose and assess optimization approaches for different applications.

Course Information

  • Cross-listed as 16:960:690:01 and 16:954:694:02.
  • Thursdays, 12:10–3:10 p.m., PH-111.
  • Prerequisites: solid analysis and linear algebra, familiarity with algorithms and computational complexity, and comfort with mathematical proofs.

Assessment and Project

One quiz contributes 20% of the grade and the course project contributes 80%. Five optional take-home questions can each add up to 5% bonus credit.

Students may work alone or in a group of two or three. Projects may develop an optimization algorithm, analyze one theoretically, or implement methods for an application. Students may select an instructor-provided topic or propose their own after discussion with the course team.

Tentative Weekly Outline

Week / DateTopic
1 · Jan 22Optimization models, examples, and constrained versus unconstrained problems
2 · Jan 29Convexity: sets, functions, strong convexity, smoothness, and geometry
3 · Feb 5Stochastic optimization, empirical risk minimization, stability, and generalization
4 · Feb 12Stochastic gradient descent: convergence, sample complexity, stability, and implementation
5 · Feb 19Coordinate descent and Frank–Wolfe methods
6 · Feb 26Mirror descent, Bregman divergence, nonsmoothness, and hidden convexity
7 · Mar 5Deep-learning optimizers and variance-reduced gradients: Adam-mini, SARAH, SPIDER, STORM, and PAGE
8 · Mar 12Matrix optimizers, including Muon variants
9 · Mar 19Spring break
10 · Mar 26Nonconvex optimization landscapes and convergence to stationary points
11 · Apr 2Stochastic minimax, bilevel, and contextual optimization; biased gradients
12 · Apr 9Distributionally robust optimization, dual formulations, computation, and generalization
13 · Apr 16Causal learning as optimization: invariance, instrumental variables, and DAG learning
14 · Apr 23Policy optimization in reinforcement learning and function approximation
15 · Apr 30Variational inference, reinforcement learning from human feedback, and fine-tuning

Reading Material

← All Courses