Academics
/
Courses
/
Descriptions
IEMS 351: Optimization Methods in Data Science

Prerequisites

IEMS 303 and 313; programming course

Description

This course offers an introduction to mathematical nonlinear optimization with applications in data science. The theoretical foundation and the fundamental algorithms for nonlinear optimization are studied and applied to supervised learning models, including nonlinear regression, logistic regression, support vector machines, and deep neural networks. Students write their own implementation of the algorithms in the Python programming language and explore their performance on realistic data sets.

This course counts as an IE/OR elective for the Industrial Engineering degree.

Learning Objectives

Students will be able to mathematically characterize optimal solutions for nonlinear optimization models
Students will know the fundamental algorithms for unconstrained optimization
Students will know how to specialize these algorithms for data science problems
Students will be able to implement these algorithms in Python
Students will be familiar with nonlinear regression, logistic regression, deep neural network, and support vector machine models
Students will have practical experience in training these models using their own Python implementations
Students will be able to write well documented and structured Python programs

Topics

Unconstrained nonlinear optimization theory and algorithms
Linear and nonlinear regression, logistic regression
Numerical solution of linear systems
Stochastic gradient descent
Deep neural networks
Introduction to constrained nonlinear optimization
Quadratic programs (example: support vector machines)
Python programming (throughout the course)

Materials

Recommended:

Optimization in Operations Research, second edition, Ronald L. Rardin, ISBN-13: 978-0-13-438455-9
An Introduction to Statistical Learning with Applications in R, Gareth James, Daniela Witten, Trevor Hastie, and Rovert Tibshirani, ISBN-13: 978-1461471370 (also available for free online)
Scientific Computing, second edition, Michael T. Heat, ISBN-13: 978-1259002281

Additional information

Homework problems include the implementation of algorithms in Python. Students are expected to familiarize themselves with basic Python programming in the first week of classes at the latest and to continue to learn new concepts throughout the course (a basic tutorial will be provided). Students should review multivariate calculus and linear algebra before the start of the course.