Title: Efficient Data-driven Methods for Robust Sequential Decision Making

Abstract: Markov decision processes involve a decision-maker seeking the optimal policy to sequentially interact with an environment in order to complete a certain task. While such a paradigm has brought substantial empirical successes, it is also known that the optimized policy often exhibits brittle robustness against potential environment disturbances. Hence when exploring its applications in critical domains such as healthcare engineering, it is imperative to ensure robust and trustworthy decisions are being made.

In this talk, we attempt to tackle this problem under the framework of robust Markov decision processes, formulated as a minimax game between the decision-maker and an adversarially changing environment. We reveal the underlying pseudo-convexity of the non-convex, non-smooth objective of the decision-maker. Consequently, we introduce a novel first-order method that achieves optimal iteration and sample complexities for the first time in the literature. While constructing the first-order information, we address the problem of evaluating the worst-case performance of a given policy. In particular, we exploit the dynamic nature of this non-concave problem and propose a globally convergent, model-free method with optimal performances. We also discuss its natural variant capable of incorporating function approximation to handle large-scale problems, thereby addressing an important yet unresolved question for robust Markov decision processes.

Biography: Yan Li is a PhD student at the School of Industrial and Systems Engineering, Georgia Institute of Technology. Yan's research centers around the algorithmic and theoretical foundations of data science. Much of his recent effort has been devoted to sequential decision-making problems and their robust counterparts. His research has been recognized by the Alice and John Jarvis Ph.D. Student Research award, the Margaret and Stephen Kendrick Research Excellence award, and the IDEaS-TRIAD fellowship from Georgia Tech.