IEMS 395-1: Optimization Methods In Data Science

Quarter Offered

Spring : MWF 2:00-2:50 (Lab: W 9:00, 10:00) ; Waechter


IEMS 303 and 313; EECS 111


This course offers an introduction to nonlinear mathematical optimization with applications in data science.  The theoretical foundation and the fundamental algorithms for nonlinear optimization are studied and applied to supervised learning models, including nonlinear regression, logistic regression, and deep neural networks.  Students write their own implementation of the algorithms in the Python programming language and explore their performance on realistic data sets.

  • This course counts as an IE/OR elective for the Industrial Engineering degree.


  • Students will be able to mathematically characterize optimal solutions for nonlinear optimization models
  • Students will know the fundamental algorithms for unconstrained optimization
  • Students will know how to specialize these algorithms for data science problems
  • Students will be able to implement these algorithms in Python
  • Students will be familiar with nonlinear regression, logistic regression, deep neural network, and support vector machine models
  • Students will have practical experience in training these models using their own Python implementations
  • Students will be able to write well documented and structured Python programs


  • Unconstrained nonlinear optimization theory and algorithms for smooth objective functions
  • Linear and nonlinear regression, logistic regression
  • Computer arithmetic and numerical solution of linear systems
  • Stochastic gradient descent
  • Deep neural networks
  • Introduction to constrained nonlinear optimization theory
  • Quadratic programs (example: support vector machines)
  • Python programming (throughout the course)



  • Optimization in Operations Research, second edition, Ronald L. Rardin, ISBN-13: 978-0-13-438455-9
  • An Introduction to Statistical Learning with Applications in R, Gareth James, Daniela Witten, Trevor Hastie, and Rovert Tibshirani, ISBN-13: 978-1461471370 (also available for free online)
  • Scientific Computing, second edition, Michael T. Heat, ISBN-13: 978-1259002281


Homework problems include the implementation of algorithms in Python. Students are expected to familiarize themselves with basic Python programming in the first week of classes at the latest and to continue to learn new concepts throughout the course (a basic tutorial will be provided). Students are also encouraged to review multivariate calculus and linear algebra before the start of the course.