News & Events
Department Events

Events

  • Nov
    5

    CANCELLED - IEMS Seminar: Adaptive Sequential SAA as a Universal Paradigm for Stochastic Optimization

    Department of Industrial Engineering and Management Sciences

    11:00 AM M228, Technological Institute

    EVENT DETAILS

    Raghu Pasupathy
    Purdue University

    Title: Adaptive Sequential SAA as a Universal Paradigm for Stochastic Optimization

    Abstract: The Stochastic Approximation (SA) recursion, also known as Stochastic Gradient Descent (SGD), is the ``workhorse" recursion for solving stochastic optimization problems arising in machine learning and elsewhere. For smooth, strongly convex objectives, it is well-known that SA iterates achieve the information theoretic Cramer-Rao (CR) lower bound when the step size is chosen correctly. Such correct choice, however, depends on unknown curvature constants; moreover, the CR bound as a measure of performance ignores transient evolution and is thus not fully reflective of finite-time performance. This explains why SGD is rarely implemented in its native form. Instead, heuristic variations, e.g., ADAM, have come to be widely preferred (see, for example, TensorFlow). In this talk, I will argue for an alternative to SGD and its variants. Specifically, I will re-introduce a decades-old idea and rigorize it in a way that allows well-established deterministic optimization methods, e.g., line search and trust-region with L-BFGS, to be used ``as is." The power of the framework is that it naturally retains the structure that is often inherent to sample-paths, and is expressly equipped to exploit deterministic optimization methods. I will demonstrate that the proposed framework retains the CR lower bound to within a factor in the convex smooth case, and an analogous theorem for the smooth non-convex case. An extension to the non-smooth convex context using the proximal method as a solver will be evident. I will also introduce two solvers, RA-LLBFGS and RA-TLBFGS, as specific variations of the framework.

    Biography: Raghu Pasupathy has been Associate Professor in the Department of Statistics at Purdue University since 2014. His recent work has focused on the theoretical and computational aspects of stochastic optimization, statistical modeling, and Monte Carlo. His teaching interests include probability and stochastic processes, statistics (especially modeling), and optimization. Raghu Pasupathy currently serves as the President of the INFORMS SImulation Society.

    more less

    TIME Tuesday, November 5, 2019 at 11:00 AM - 12:00 PM

    LOCATION M228, Technological Institute    map it

    ADD TO CALENDAR

    CONTACT Agnes Kaminski    a-kaminski@northwestern.edu EMAIL

    CALENDAR Department of Industrial Engineering and Management Sciences

  • May
    8

    Extending Care: A Conversation about Conservation and Futurity

    Block Museum of Art

    6:00 PM Block Museum of Art, Mary and Leigh

    EVENT DETAILSmore info

    TIME Wednesday, May 8, 2024 at 6:00 PM - 7:30 PM

    LOCATION Block Museum of Art, Mary and Leigh    map it

    ADD TO CALENDAR

    CONTACT Block Museum of Art    block-museum@northwestern.edu EMAIL

    CALENDAR Block Museum of Art

  • May
    14

    Letting the Samples Speak: A New Approach for Importance Sampling for Tail Events

    Department of Industrial Engineering and Management Sciences (IEMS)

    11:00 AM ITW 1.350, Ford Motor Company Engineering Design Center

    EVENT DETAILS

    TIME Tuesday, May 14, 2024 at 11:00 AM - 12:00 PM

    LOCATION ITW 1.350, Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Kendall Minta    kendall.minta@gmail.com EMAIL

    CALENDAR Department of Industrial Engineering and Management Sciences (IEMS)

  • Jun
    10

    Northwestern Engineering PhD Hooding and Master's Degree Recognition Ceremony

    McCormick School of Engineering and Applied Science

    9:00 AM Welsh-Ryan Arena

    EVENT DETAILSmore info

    TIME Monday, June 10, 2024 at 9:00 AM - 11:00 AM

    LOCATION Welsh-Ryan Arena   

    ADD TO CALENDAR

    CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Jun
    10

    Northwestern Engineering Undergraduate Convocation

    McCormick School of Engineering and Applied Science

    2:00 PM Welsh-Ryan Arena

    EVENT DETAILSmore info

    TIME Monday, June 10, 2024 at 2:00 PM - 4:00 PM

    LOCATION Welsh-Ryan Arena   

    ADD TO CALENDAR

    CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Aug
    14

    Undergraduate Quantum Summer School

    McCormick School of Engineering and Applied Science

    All Day Ford Motor Company Engineering Design Center

    EVENT DETAILSmore info

    TIME Wednesday, August 14, 2024

    LOCATION Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Dongyang Li    lidongyang@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Aug
    15

    Undergraduate Quantum Summer School

    McCormick School of Engineering and Applied Science

    All Day Ford Motor Company Engineering Design Center

    EVENT DETAILSmore info

    TIME Thursday, August 15, 2024

    LOCATION Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Dongyang Li    lidongyang@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science