News & Events
Department Events

Events

  • Nov
    5

    CANCELLED - IEMS Seminar: Adaptive Sequential SAA as a Universal Paradigm for Stochastic Optimization

    Department of Industrial Engineering and Management Sciences

    11:00 AM M228, Technological Institute

    EVENT DETAILS

    Raghu Pasupathy
    Purdue University

    Title: Adaptive Sequential SAA as a Universal Paradigm for Stochastic Optimization

    Abstract: The Stochastic Approximation (SA) recursion, also known as Stochastic Gradient Descent (SGD), is the ``workhorse" recursion for solving stochastic optimization problems arising in machine learning and elsewhere. For smooth, strongly convex objectives, it is well-known that SA iterates achieve the information theoretic Cramer-Rao (CR) lower bound when the step size is chosen correctly. Such correct choice, however, depends on unknown curvature constants; moreover, the CR bound as a measure of performance ignores transient evolution and is thus not fully reflective of finite-time performance. This explains why SGD is rarely implemented in its native form. Instead, heuristic variations, e.g., ADAM, have come to be widely preferred (see, for example, TensorFlow). In this talk, I will argue for an alternative to SGD and its variants. Specifically, I will re-introduce a decades-old idea and rigorize it in a way that allows well-established deterministic optimization methods, e.g., line search and trust-region with L-BFGS, to be used ``as is." The power of the framework is that it naturally retains the structure that is often inherent to sample-paths, and is expressly equipped to exploit deterministic optimization methods. I will demonstrate that the proposed framework retains the CR lower bound to within a factor in the convex smooth case, and an analogous theorem for the smooth non-convex case. An extension to the non-smooth convex context using the proximal method as a solver will be evident. I will also introduce two solvers, RA-LLBFGS and RA-TLBFGS, as specific variations of the framework.

    Biography: Raghu Pasupathy has been Associate Professor in the Department of Statistics at Purdue University since 2014. His recent work has focused on the theoretical and computational aspects of stochastic optimization, statistical modeling, and Monte Carlo. His teaching interests include probability and stochastic processes, statistics (especially modeling), and optimization. Raghu Pasupathy currently serves as the President of the INFORMS SImulation Society.

    more less

    TIME Tuesday, November 5, 2019 at 11:00 AM - 12:00 PM

    LOCATION M228, Technological Institute    map it

    ADD TO CALENDAR

    CONTACT Agnes Kaminski    a-kaminski@northwestern.edu EMAIL

    CALENDAR Department of Industrial Engineering and Management Sciences

  • Sep
    23

    Welcome & Luncheon for New Full-time Graduate Students

    McCormick School of Engineering and Applied Science

    11:00 AM Ryan Auditorium, Technological Institute

    EVENT DETAILS

    TIME Monday, September 23, 2024 at 11:00 AM - 12:30 PM

    LOCATION Ryan Auditorium, Technological Institute    map it

    ADD TO CALENDAR

    CONTACT Andi Joppie    andi.joppie@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Sep
    24

    Fall 2024 Classes Begin (No Northwestern Monday in Fall)

    University Academic Calendar

    All Day

    EVENT DETAILS

    TIME Tuesday, September 24, 2024

    ADD TO CALENDAR

    CONTACT Office of the Registrar    nu-registrar@northwestern.edu EMAIL

    CALENDAR University Academic Calendar

  • Sep
    30

    Last day to add a class or change a section for Fall (Last day for tuition adjustment related to enrollment changes (to or from full-time). No reductions are made to bills for dropped or swapped classes after this date)

    University Academic Calendar

    All Day

    EVENT DETAILS

    TIME Monday, September 30, 2024

    ADD TO CALENDAR

    CONTACT Office of the Registrar    nu-registrar@northwestern.edu EMAIL

    CALENDAR University Academic Calendar

  • Dec
    7

    Fall classes end

    University Academic Calendar

    All Day

    EVENT DETAILS

    TIME Saturday, December 7, 2024

    ADD TO CALENDAR

    CONTACT Office of the Registrar    nu-registrar@northwestern.edu EMAIL

    CALENDAR University Academic Calendar