IEMS 459: Convex Optimization

Quarter Offered

None ; Nocedal


Linear Algebra, Calculus , Real Analysis


The goal of this course is to investigate in-depth and to develop expert knowledge in the theory and algorithms for convex optimization. This course will provide a rigorous introduction to the rich field of convex analysis, particularly as it relates to mathematical optimization and duality theory. In addition to formal analytical tools and concepts, emphasis will be placed on developing a geometric and intuitive understanding of convex objects, optimization problems, and duality concepts. The course will focus on practical algorithms.

  • This is not a required class. It is intended to be an advanced PhD optimization course


  • Students will understand the fundamental properties of convex sets and functions
  • Students will understand the central role of convexity in optimization
  • Students will learn the basic mechanisms that drive convergence of optimization algorithms
  • Students will gain fundamental understanding of duality via insights provided by geometric arguments
  • Students will learn the scientific tools that are relevant for different classes of optimization problems and different problem sizes
  • Students will learn to implement optimization algorithms


  • Convex Sets; Convex Functions
  • Cones; Subgradients
  • First-order methods
    • Gradient method
    • Gradient projection
    • Proximal gradient
    • Coordinate descent
    • Stochastic gradient
    • Second-order methods
    • Applications in Statistics
    • Duality
      • Lagrangian Duality
      • Fenchel Duality
      • Dual Methods
      • Stochastic dual coordinate descent methods
      • Non-smooth optimization
        • Subgradient methods
        • Cutting plane and Bundle methods
        • Quasi-Newton and Trust-region methods
        • Gradient Sampling methods
        • Augmented Lagrangian method and ADMM
        • Frank Wolfe method


  • Required: (none)
  • Recommended:
    • o Convex Optimization Algorithms, Dimitri P. Bertsekas, ISBN: 978-1886529281