Inside Our ProgramProgram Events
Events
-
Apr19
EVENT DETAILS
less
Friday / CS Seminar
April 19th / 12:00 PM
In Person / Mudd 3514Speaker
Dan Fu, Stanford UniversityTalk Title
Hardware-Aware Efficient Primitives for Machine LearningAbstract
Efficiency is increasingly tied to quality to machine learning, with more efficient training algorithms leading to more powerful models. However, today's most popular machine learning models are built on asymptotically inefficient primitives. For example, attention in Transformers scales quadratically in the input size, while MLPs scale quadratically in model dimension. In this talk, I discuss my work on improving the efficiency of the core primitives in machine learning, with an emphasis on hardware-aware algorithms and long-context applications. First, I focus on replacing attention with gated state space models (SSMs) and convolutions, which scale sub-quadratically in context length. I describe the H3 (Hungry Hungry Hippos) architecture, a gated SSM architecture that matches Transformers in quality up to 3B parameters and achieves 2.4x faster inference. Second, I focus on developing hardware-aware algorithms for SSMs and convolutions. I describe FlashFFTConv, a fast algorithm for computing SSMs and convolutions on GPU by optimizing the Fast Fourier Transform (FFT). FlashFFTConv yields up to 7x speedup and 5x memory savings, even over vendor solutions from Nvidia. Third, I will briefly touch on how these same techniques can also be used to develop sub-quadratic scaling in the model dimension. I will describe Monarch Mixer, which uses a generalization of the FFT to achieve sub-quadratic scaling in both sequence length and model dimension. Throughout the talk, I will give examples of how these ideas are beginning to take hold, with gated SSMs and their variants now leading to state-of-the-art performance in long-context language models, embedding models, and DNA foundation models.Biography
Dan Fu is a PhD student in the Computer Science Department at Stanford University, where he is co-advised by Christopher Ré and Kayvon Fatahalian. His research interests are at the intersection of systems and machine learning. Recently, he has focused on developing algorithms and architectures to make machine learning more efficient, especially for enabling longer-context applications. His research has appeared as oral and spotlight presentations at NeurIPS, ICML, and ICLR, and he has received the best student paper runner up at UAI. Dan has also been supported by an NDSEG fellowship.Research Area/Interests
machine learning, systemsTIME Friday, April 19, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May1
EVENT DETAILS
lessIt looks like AI is here to stay. Not only that, its impact on society will rival or exceed that of the world wide web. However, we are likely just scratching the surface and AI is still in its infancy. The expected growth is immense and there is an active arms race in both hardware and algorithms. Nvidia dominates hardware today, largely because we are still in the training era, where time-to-accuracy at any cost rules. In this era, algorithmic innovations offer the biggest benefits and require a robust programmable platform (i.e., Nvidia GPUs) upon which to innovate. However, there are signs the inference era is imminent, where the rules are different. The projected scales of AI inference deployments will demand highly efficient hardware. With AI touching all layers of the computing stack today, we can leverage algorithm-hardware co-design to uncover interesting opportunities and trends for efficiency via customization and acceleration. This talk presents an overview of a NSF-funded project we had proposed to develop a cloudless universal translator, which we thought eventually possible given the state of ML technology at the time. Throughout
the project, we designed three custom AI SoCs--targeting automatic speech recognition (FlexASR), natural language processing (EdgeBERT), and on-device learning/fine-tuning (CAMEL). These designs highlight ways to maximize energy efficiency given the probabilistic nature of machine learning algorithms. Finally, the talk closes with some thoughts on
chiplets and carbon.TIME Wednesday, May 1, 2024 at 11:00 AM - 12:00 PM
LOCATION L440, Technological Institute map it
CONTACT Catherine Healey catherine.healey@northwestern.edu EMAIL
CALENDAR Department of Electrical and Computer Engineering (ECE)
-
May1
EVENT DETAILS
lessWednesday / CS Distinguished Lecture
May 1st / 12:00 PM
Hybrid / Mudd 3514Speaker
Tong Zhang, University of Illinois Urbana-ChampaignTalk Title
Machine Learning Methods for Finetuning and Alignment of Large Language ModelsAbstract
Large Language Models (LLMs) represent a significant milestone in the development of artificial general intelligence. Numerous pretrained models are available within the open-source community, yet they often require further training to be effectively utilized in downstream applications. The enhancement of these models is primarily accomplished through two methods: finetuning and Reinforcement Learning from Human Feedback (RLHF). Finetuning is crucial for tailoring LLMs to specialized topics or for developing capabilities such as instruction following. Meanwhile, RLHF aims to align LLMs with human preferences.
This talk presents various machine learning problems and algorithms for both finetuning and RLHF. For finetuning, we will introduce techniques that minimize hallucinations and enhance resource efficiency in optimization processes. For RLHF, we will explore both the theoretical frameworks that establish principled approaches and the practical algorithms that implement these theories. The presentation highlights some key questions and solutions in optimizing LLMs for enhanced functionality and alignment with human preferences.
Biography
Tong Zhang is currently a professor in the Computer Science department at the University of Illinois Urbana Champaign. He is a fellow of the IEEE, American Statistical Association, and Institute of Mathematical Statistics. His research interests include machine learning theory, algorithms, and applications. Tong Zhang has served as the chair or area chair in major machine learning conferences such as NeurIPS, ICML, and COLT, and has also served on the editorial boards of leading machine learning journals such as PAMI, JMLR, and the Machine Learning Journal.Research Area/Interests
TBAZoom: TBA
TIME Wednesday, May 1, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May8
EVENT DETAILS
lessWednesday / CS Distinguished Lecture
May 8th / 12:00 PM
Hybrid / Mudd 3514Speaker
Monica Lam, Stanford UniversityTalk Title
TBAAbstract
TBABiography
TBAResearch Area/Interests
TBAZoom
TBATIME Wednesday, May 8, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May9
EVENT DETAILSmore info
lessWant to join the decision-makers? Register for Northwestern University's Master of Science in Information Technology (MSIT) Program Information Session and learn how you can enhance your knowledge of IT and gain the business management skills you need to direct effective IT strategy.
TIME Thursday, May 9, 2024 at 6:00 PM - 7:00 PM
CONTACT Svetlana Korzeniowski msit@northwestern.edu EMAIL
CALENDAR MS in Information Technology (MSIT) Program
-
May10
EVENT DETAILS
lessFriday / CS Distinguished Lecture
May 10th / 12:00 PM
Hybrid / Mudd 3514Speaker
Shafi Goldwasser; Simons Institute for the Theory of Computing, University of California BerkeleyTalk Title
Trust, Backdoor Vulnerabilities and Possible MitigationsAbstract
Cryptographic tools and models enable to trust the use of technology platforms controlled by worst case computationally bounded adversaries. In this talk I will use cryptographic modeling and tools to view trust dilemmas in various phases of the machine learning pipelines. We will touch on privacy in the training stage, verification of properties of
machine learning models, and the possibility of achieving robustness in presence of backdoors.Biography
Shafi Goldwasser is Director of the Simons Institute for the Theory of Computing, and Professor of Electrical Engineering and Computer Science at the University of California Berkeley. Goldwasser is also the RSA Professor (post tenure) of Electrical Engineering and Computer Science at MIT and Professor Emeritus of Computer Science and Applied Mathematics at the Weizmann Institute of Science, Israel. Goldwasser holds a B.S. Applied Mathematics from Carnegie Mellon University (1979), and M.S. and Ph.D. in Computer Science from the University of California Berkeley (1984).
Goldwasser's contributions include the introduction of probabilistic encryption, interactive zero knowledge protocols, elliptic curve primality testings, hardness of approximation proofs for combinatorial problems, combinatorial property testing, and pseudo deterministic algorithms.
Goldwasser was the recipient of the ACM Turing Award in 2012, the Gödel Prize in 1993 and in 2001, the ACM Grace Murray Hopper Award in 1996, the RSA Award in Mathematics in 1998, the ACM Athena Award for Women in Computer Science in 2008, the Benjamin Franklin Medal in 2010, the IEEE Emanuel R. Piore Award in 2011, the Simons Foundation Investigator Award in 2012, and the BBVA Foundation Frontiers of Knowledge Award in 2018. Goldwasser is a member of the NAS, NAE, AAAS, the Russian Academy of Science, the Israeli Academy of Science, the London Royal Mathematical Society and a Foreign Member of the Royal Society. Goldwasser holds honorary degrees from Ben Gurion University, Bar Ilan University, Carnegie Mellon University, Haifa University, University of Oxford, and the University of Waterloo, and has received the UC Berkeley Distinguished Alumnus Award and the Barnard College Medal of Distinction.
Research Area/Interests
Cryptography, Complexity, Probabilistic AlgorithmsZoom: TBA
TIME Friday, May 10, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May30
EVENT DETAILS
lessTBA
TIME Thursday, May 30, 2024 at 9:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May30
EVENT DETAILS
lessTBA
TIME Thursday, May 30, 2024 at 3:00 PM - 5:00 PM
LOCATION TBA, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
