Please join us in the Electrical and Computer Engineering Department at the Technological Institute for an hour-long Meet the Faculty seminar with Professor Stephen Xia.

Student Introduction by Jordan Athas.

Abstract: Large language models are becoming increasingly relevant to applications across smart cities, wearables, healthcare, and other areas of computing. Supporting these applications on resource-constrained edge platforms requires considering dimensions that are impacted by all layers of the computing stack, such as resource availability, latency, and energy constraints. However, existing approaches typically optimize just a single layer of the stack at a time, such as processor frequency, model architecture, or decoding policy, which leaves important cross-layer optimizations underexplored.

This talk presents several efforts toward efficient LLM inference by optimizing multiple layers of the stack at runtime. First, we introduce our work on co-adapting both the model and underlying system to enable real-time and energy efficient LLM decoding for edge platforms. This work controls not only hardware and operating system level parameters, such as processor frequency, but also adapts the underlying model itself through speculative decoding, which leverages a smaller "draft" model in place of the LLM to predict multiple tokens into the future. Next, we discuss how the performance of speculative decoding can be enhanced along two complementary axes, time and depth, to achieve further speed ups and open new directions for efficient and robust AI inference.

Bio: Stephen Xia is an assistant professor in the Department of Electrical and Computer Engineering and (by courtesy) Computer Science at Northwestern University. His research lies at the intersection between systems, embedded machine learning, and data, spanning areas in mobile and embedded computing, Internet-of-Things, cyber-physical systems, artificial intelligence, and digital health. His work focuses on enabling intelligent perception, reasoning, and understanding on common objects found all around us to create more intelligent and autonomous environments. Stephen's research has been highlighted by many popular media outlets and received multiple best demo, presentation, and paper awards. Prior to Northwestern, Stephen was a postdoctoral scholar at UC Berkeley, received his Ph.D. from Columbia University, and received his B.S. from Rice University, all in Electrical Engineering.