Abstract

This talk consists of two parts. In the first part, we study the inference reliability of large language models (LLMs), with an emphasis on characterizing how prompt design affects the accuracy and stability of model outputs. We discuss recent theoretical and empirical advances in in-context learning and hallucination, and examine conditions under which LLMs produce reliable predictions. In the second part, we consider constrained bilevel optimization and develop efficient algorithms with provable guarantees. In particular, we show how error bound conditions can be leveraged to obtain fast convergence rates under mild assumptions.

Bio

Jiawei Zhang is an assistant professor in the Department of Computer Sciences at the University of Wisconsin-Madison. Prior to joining UW-Madison, he was a postdoctoral researcher in the Laboratory for Information & Decision Systems (LIDS) at the Massachusetts Institute of Technology, working with Asuman Ozdaglar and Saurabh Amin. He received his PhD in Computer and Information Engineering from the Chinese University of Hong Kong, Shenzhen under the supervision of Zhi-Quan Luo. He earned his BSc in Mathematics (Hua Loo-Keng Talent Program) from the University of Science and Technology of China.

His research focuses on optimization theory and algorithms, the theoretical foundations and algorithmic development of generative models (including large language models and diffusion models), and optimization methods for engineering applications such as energy systems, communications, and manufacturing.