IEMS 351: Optimization Methods In Data Science

Quarter Offered

None : TBA ; Wächter


IEMS 303 and 313; programming course


This course offers an introduction to mathematical nonlinear optimization with applications in data science.  The theoretical foundation and the fundamental algorithms for nonlinear optimization are studied and applied to supervised learning models, including nonlinear regression, logistic regression, support vector machines, and deep neural networks.  Students write their own implementation of the algorithms in the Python programming language and explore their performance on realistic data sets.

  • This course counts as an IE/OR elective for the Industrial Engineering degree.

 Learning Objectives

  • Students will be able to mathematically characterize optimal solutions for nonlinear optimization models
  • Students will know the fundamental algorithms for unconstrained optimization
  • Students will know how to specialize these algorithms for data science problems
  • Students will be able to implement these algorithms in Python
  • Students will be familiar with nonlinear regression, logistic regression, deep neural network, and support vector machine models
  • Students will have practical experience in training these models using their own Python implementations
  • Students will be able to write well documented and structured Python programs


  • Unconstrained nonlinear optimization theory and algorithms
  • Linear and nonlinear regression, logistic regression
  • Numerical solution of linear systems
  • Stochastic gradient descent
  • Deep neural networks
  • Introduction to constrained nonlinear optimization
  • Quadratic programs (example: support vector machines)
  • Python programming (throughout the course)



  • Optimization in Operations Research, second edition, Ronald L. Rardin, ISBN-13: 978-0-13-438455-9
  • An Introduction to Statistical Learning with Applications in R, Gareth James, Daniela Witten, Trevor Hastie, and Rovert Tibshirani, ISBN-13: 978-1461471370 (also available for free online)
  • Scientific Computing, second edition, Michael T. Heat, ISBN-13: 978-1259002281


IEMS 303 and 313; programming course

Additional information

Homework problems include the implementation of algorithms in Python. Students are expected to familiarize themselves with basic Python programming in the first week of classes at the latest and to continue to learn new concepts throughout the course (a basic tutorial will be provided). Students should review multivariate calculus and linear algebra before the start of the course.