Optimization for Machine Learning

COM S 578X

Offered during Fall Semester each year.

  1. Credits: 3 credit hours
  2. Instructor's or course coordinator's name: Prof. Jia (Kevin) Liu
  3. Textbook, title, author, and year
    • S. Boyd and L. Vandenberghe, "Convex Optimization," Cambridge University Press, 2004.
    • Y. Nesterov, "Introductory Lectures on Convex Optimization: A Basic Course," Springer, 2004.
    • M. Bazarra, H.D. Sherali, and C.M. Shetty, "Nonlinear Programming: Theory and Algorithms," John Wiley and Sons, 2006.
    • J. Nocedal and Stephen J. Wright, "Numerical Optimization," Springer, 2006.
  4. Other supplemental materials: None

Course Information

  1. Brief description of the content of the course: Advances in optimization theory and algorithms with evolving applications for machine learning. Theoretical foundations at the intersection of optimization and machine learning to conduct advanced research in machine learning and related fields. Emphasis on proof techniques for optimization algorithms in machine learning.
  2. Prerequisites or co-requisites: COM S 472, COM S 474


  1. Fundamentals of convex analysis
  2. First-order methods
  3. Stochastic first-order methods
  4. Sparse/regularized optimization
  5. Proximal and operator splitting
  6. Nonconvex optimization in machine learning