News & Events
Department Events

Events

  • Apr
    9

    Constant Stepsize Stochastic Methods in Min-Max and Variational Inequality Problems

    Department of Industrial Engineering and Management Sciences (IEMS)

    11:00 AM A230, Technological Institute

    EVENT DETAILS

    Abstract: Many reinforcement/machine learning problems involve loss minimization, min-max optimization and fixed-point equations, all of which can be cast under the framework of Variational Inequalities (VIs). Stochastic methods like SGD, SEG and TD/Q Learning are prevalent, and their constant stepsize versions have gained popularity due to effectiveness and robustness. Viewing the iterates of these algorithms as a Markov chain, we study their fine-grained probabilistic behavior. In particular, we establish finite-time geometric convergence of the iterates distribution, and relate the ergodicity properties of the Markov chain to the characteristics of the VI, algorithm and data.

    Using techniques of coupling and basic adjoint relationship, we characterize the limit distribution and how its bias depends on the stepsize. For smooth problems, exemplified by TD learning and smooth min-max optimization, the bias is proportional to the stepsize. For nonsmooth problems, exemplified by Q-learning and generalized linear model with nonsmooth link functions (e.g., ReLU), the bias has drastically different behavior and scales with the square root of the stepsize.

    This precise probabilistic characterization allows for variance reduction via tail-averaging and bias reduction via Richardson-Romberg extrapolation. The combination of constant stepsize, averaging and extrapolation provides a favorable balance between fast mixing and low long-run error, and we demonstrate its effectiveness in statistical inference compared to traditional diminishing stepsize schemes.

    Bio: Qiaomin Xie is an assistant professor in the Department of Industrial and Systems Engineering at the University of Wisconsin-Madison. Her research interests lie in the fields of reinforcement learning, applied probability, game theory and stochastic networks, with applications to computer and communication networks. She was previously a visiting assistant professor at School of Operations Research and Information Engineering at Cornell University (2019-2021). Prior to that, she was a postdoctoral researcher with LIDS at MIT. Qiaomin received her Ph.D. in Electrical and Computing Engineering from University of Illinois Urbana-Champaign in 2016. She received her B.S. in Electronic Engineering from Tsinghua University. She is a recipient of NSF CAREER Award, JPMorgan Faculty Research Award, Google Systems Research Award and UIUC CSL PhD Thesis Award.

    more less

    TIME Tuesday, April 9, 2024 at 11:00 AM - 12:00 PM

    LOCATION A230, Technological Institute    map it

    ADD TO CALENDAR

    CONTACT Kendall Minta    kendall.minta@gmail.com EMAIL

    CALENDAR Department of Industrial Engineering and Management Sciences (IEMS)

  • May
    20

    Curious Life Certificate Information Session

    McCormick School of Engineering and Applied Science

    12:00 PM Design Studio 3 (Room 3-210), Ford Motor Company Engineering Design Center

    EVENT DETAILS

    TIME Monday, May 20, 2024 at 12:00 PM - 1:30 PM

    LOCATION Design Studio 3 (Room 3-210), Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Bruce Ankenman    ankenman@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Jun
    10

    Northwestern Engineering PhD Hooding and Master's Degree Recognition Ceremony

    McCormick School of Engineering and Applied Science

    9:00 AM Welsh-Ryan Arena

    EVENT DETAILSmore info

    TIME Monday, June 10, 2024 at 9:00 AM - 11:00 AM

    LOCATION Welsh-Ryan Arena   

    ADD TO CALENDAR

    CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Jun
    10

    Northwestern Engineering Undergraduate Convocation

    McCormick School of Engineering and Applied Science

    2:00 PM Welsh-Ryan Arena

    EVENT DETAILSmore info

    TIME Monday, June 10, 2024 at 2:00 PM - 4:00 PM

    LOCATION Welsh-Ryan Arena   

    ADD TO CALENDAR

    CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Aug
    14

    Undergraduate Quantum Summer School

    McCormick School of Engineering and Applied Science

    All Day Ford Motor Company Engineering Design Center

    EVENT DETAILSmore info

    TIME Wednesday, August 14, 2024

    LOCATION Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Dongyang Li    lidongyang@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science

  • Aug
    15

    Undergraduate Quantum Summer School

    McCormick School of Engineering and Applied Science

    All Day Ford Motor Company Engineering Design Center

    EVENT DETAILSmore info

    TIME Thursday, August 15, 2024

    LOCATION Ford Motor Company Engineering Design Center    map it

    ADD TO CALENDAR

    CONTACT Dongyang Li    lidongyang@northwestern.edu EMAIL

    CALENDAR McCormick School of Engineering and Applied Science