Strong Northwestern Presence at the 2024 NeurIPS Conference

Northwestern Engineering has a strong presence at the Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS), being held December 10-15 in Vancouver.

NeurIPS is the premier annual forum for interdisciplinary research in fields including artificial intelligence, computational neuroscience, computer vision, machine learning, natural language processing, optimization, reinforcement learning, and statistics.

Northwestern’s Minshuo Chen, Matthew Groh, Aggelos Katsaggelos, Manling Li, Han Liu, Miklos Racz, Aravindan Vijayaraghavan, Xiao Wang, Zhaoran Wang, Xinyu Xing, Qi Zhu, and several of their current and former lab members will present impactful research at the event.

“I am delighted to see such a strong presence of papers from Northwestern faculty,” said Samir Khuller, Peter and Adrienne Barris Chair of Computer Science at the McCormick School of Engineering. “Our expertise in machine learning and related areas spans multiple departments on campus. This allows us to work together to offer many opportunities to our graduate students, who can come to Northwestern and collaborate with researchers across domains including computer science, computer engineering, industrial engineering, statistics, and data science.”

NeurIPS 2024 Conference

Manling Li, an assistant professor of computer science who joined Northwestern Engineering this fall, will present her team’s systematic evaluation framework — called Embodied Agent Interface — to benchmark large language models for embodied decision-making. In these choice scenarios, agents capable of following human instructions achieve specific goals through a sequence of actions in various digital and physical environments.

NeurIPS oral presentations represent less than 1 percent of the top conference submissions. The project won the Best Paper Award at the Southern California Natural Language Processing Symposium in November.

Coauthors of the paper “Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making” include Qineng Wang and Kangrui Wang, first-year PhD students in computer science at Northwestern Engineering; Jiajun Wu, Fei-Fei Li, Percy Liang, Shiyu Zhao, Yu Zhou, Ruohan Zhang, Weiyu Liu, Sanjana Srivastava, Cem Gokmen, and Tony Lee (Stanford University); Jiayuan Mao (MIT); and Li Erran Li (Amazon).

Northwestern contributions to the NeurIPS 2024 main conference also include:

• “Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks” — Minshuo Chen, assistant professor of industrial engineering and management sciences at Northwestern Engineering; Kaiqi Zhang and Yu-Xiang Wang (University of California, Santa Barbara); Zixuan Zhang and Tuo Zhao (Georgia Institute of Technology); Mengdi Wang (Princeton University); and Yuma Takeda (University of Tokyo).
• “Gradient Guidance for Diffusion Models: An Optimization Perspective” — Chen; and Yingqing Guo, Mengdi Wang, Yukang Yang, and Hui Yuan (Princeton University).
• “A Theoretical Perspective for Speculative Decoding Algorithm” — Chen; and Kaixuan Huang, Mengdi Wang, and Ming Yin (Princeton University).
• “Intrinsic Self-Supervision for Data Quality Audits” — Matthew Groh, assistant professor of management and organizations at Northwestern’s Kellogg School of Management and (by courtesy) assistant professor of computer science at Northwestern Engineering; Alvaro Gonzalez-Jimenez, Philippe Gottfrois, Fabian Gröger, and Alexander A. Navarini (University of Basel); and Ludovic Amruthalingam, Simone Lionetti, and Marc Pouly (Lucerne University of Applied Sciences and Arts).
• “Sm: Enhanced Localization in Multiple Instance Learning for Medical Imaging Classification” — Aggelos Katsaggelos, Joseph Cummings Professor of Electrical and Computer Engineering at Northwestern Engineering and (by courtesy) professor of computer science and radiology; Yunan Wu (PhD ’24); and Pablo Morales Alvarez, Francisco M. Castro-Macías, and Rafael Molina (University of Granada).
• “HourVideo: 1-Hour Video-Language Understanding” — Li; and Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Agrim Gupta, Lea M. Hadzic, Jimming He, Taran Kota, Fei-Fei Li, and Jiajun Wu (Stanford University).
• “IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos” — Li; Cristobal Eyzaguirre, Shubh Khanna, Yunong Liu, Weiyu Liu, Juan Carlos Niebles, and Jiajun Wu (Stanford University); and Saumitra Mishra and Vineeth Ravi (J.P. Morgan AI Research).
• “On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)” — Han Liu, Orrington Lunt Professor of Computer Science and (by courtesy) professor of industrial engineering and management sciences at Northwestern Engineering and professor of statistics at Northwestern’s Weinberg College of Arts and Sciences; Jerry Yao-Chieh Hu and Weimin Wu, PhD students in computer science at Northwestern Engineering; Sophia Pi, a third-year student pursuing a bachelor’s degree in computer science at Northwestern Engineering as well as a joint major in economics and mathematical methods in the social sciences through Weinberg; Zhuoru Li (Fudan University); and Zhao Song (University of California, Berkeley).
• “Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes” — Liu, Hu, and Dennis Wu, a PhD student in computer science at Northwestern Engineering.
• “Global Convergence in Training Large-Scale Transformers” and “One-Layer Transformer Provably Learns One-Nearest Neighbor In Context” — Liu; Yuan Cao (University of Hong Kong); and Jianqing Fan, Cheng Gao, Yihan He, Jason Klusowski, Zihao Li, and Mengdi Wang (Princeton University).
• “Efficient Graph Matching for Correlated Stochastic Block Models” — Miklos Racz, assistant professor of computer science at Northwestern Engineering and professor of statistics at Weinberg; and Shuwen Chai, a PhD student in computer science at Northwestern Engineering.
• “Harnessing Multiple Correlated Networks for Exact Community Recovery” — Racz and Jifan Zhang, a PhD student in statistics and data science at Weinberg.
• “Theoretical Analysis of Weak-to-Strong Generalization” — Aravindan Vijayaraghavan, associate professor of computer science and (by courtesy) industrial engineering and management sciences at Northwestern Engineering; and Hunter Lang and David Sontag (MIT).
• “Nimbus: Secure and Efficient Two-Party Inference for Transformers” — Xiao Wang, assistant professor of computer science at Northwestern Engineering; Minyi Guo, Jingwen Leng, Zhengyi Li, and Yu Yu (Shanghai Jiao Tong University); Kang Yang (State Key Laboratory of Cryptology); and Wen-jie Lu, Jin Tan, Haoqi Wu, Derun Zhao, and Yancheng Zheng (Ant Group).
• “Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer” — Zhaoran Wang, associate professor of industrial engineering and management sciences and (by courtesy) computer science at Northwestern Engineering; Hongyi Guo, Zhihan Liu, and Shenao Zhang, PhD students in industrial engineering and management sciences at Northwestern Engineering; Jose Blanchet and Miao Lu (Stanford University); and Boyi Liu and Yingxiang Yang (ByteDance Inc.).
• “Soft-Label Integration for Robust Toxicity Classification” — Xinyu Xing, associate professor of computer science at Northwestern Engineering; Zelei Cheng and Jiahao Yu, PhD students in computer science at Northwestern Engineering; Xian Wu (PhD ’24); Shuo Han, a PhD student in statistics at Weinberg; and Xin-Qiang Cai (University of Tokyo).
• “Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval” — Qi Zhu, professor of electrical and computer engineering and (by courtesy) computer science at Northwestern Engineering; Lixu Wang, a PhD student in computer science at Northwestern Engineering; and Xinyu Du (General Motors Global Research and Development).
• “Variational Delayed Policy Optimization” — Zhu; Sinong (Simon) Zhan, a PhD student in computer engineering at Northwestern Engineering; Yixuan Wang (PhD ’24); Yuhui Wang (King Abdullah University of Science and Technology); Chung-Wei Lin (National Taiwan University); Chen Lv (Nanyang Technological University); and Chao Huang and Qingyuan Wu (University of Southampton).

NeurIPS 2024 Workshops

Jessica Hullman, Ginni Rometty Professor of Computer Science at Northwestern Engineering, is a co-organizer of the NeurIPS 2024 “Workshop on Statistical Frontiers in LLMs and Foundation Models,” one of 56 accepted workshops at this year’s conference.

This workshop will explore topics at the intersection of statistics and foundation models, including benchmarking, measuring and correcting bias, automatic evaluation, watermarking, models and data auditing, and uncertainty quantification.

Additional Northwestern contributions to the NeurIPS 2024 workshops include:

• “Unexploited Information Value in Human-AI Collaboration” at the Workshop on Behavioral Machine Learning — Hullman; Jason Hartline, professor of computer science at Northwestern Engineering; and Ziyang Guo and Yifan Wu, PhD students in computer science at Northwestern Engineering.
• “Regulation of Algorithmic Collusion, Refined: Testing Worst-case Calibrated Regret” at the Regulatable ML: Towards Bridging the Gaps between Machine Learning Research and Regulations workshop — Hartline; Chang Wang and Chenhao Zhang, PhD students in computer science at Northwestern Engineering.
• “Controlling for Unobserved Confounding with Large Language Model Classification of Patient Smoking Status” at the AIM-FM: Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond workshop — Zach Wood-Doughty, assistant professor of instruction at Northwestern Engineering; and Samuel Lee, fourth-year student in computer science at Weinberg.
• “Using Scenario-Writing for Identifying and Mitigating Impacts of Generative AI” at the Evaluating Evaluations: Examining Best Practices for Measuring Broader Impacts of Generative AI workshop — Nicholas Diakopoulos, professor of communication studies at Northwestern’s School of Communication and (by courtesy) professor of computer science at Northwestern Engineering; and Natali Helberger and Kimon Kieslich (University of Amsterdam).
• “Towards Leveraging News Media to Support Impact Assessment of AI Technologies” at the Evaluating Evaluations workshop — Diakopoulos; Mowafak Allaham, a PhD student in Northwestern's Technology and Social Behavior program; and Kimon Kieslich (University of Amsterdam).
• “Learning from Personal Preferences” at the Pluralistic Alignment Workshop — Hullman; Kelly Jiang, a PhD student in computer science at Northwestern Engineering; and Berk Ustun (University of California, San Diego).