Inside Our ProgramProgram Events
Events
-
Apr19
EVENT DETAILS
less
Friday / CS Seminar
April 19th / 12:00 PM
In Person / Mudd 3514Speaker
Dan Fu, Stanford UniversityTalk Title
Hardware-Aware Efficient Primitives for Machine LearningAbstract
Efficiency is increasingly tied to quality to machine learning, with more efficient training algorithms leading to more powerful models. However, today's most popular machine learning models are built on asymptotically inefficient primitives. For example, attention in Transformers scales quadratically in the input size, while MLPs scale quadratically in model dimension. In this talk, I discuss my work on improving the efficiency of the core primitives in machine learning, with an emphasis on hardware-aware algorithms and long-context applications. First, I focus on replacing attention with gated state space models (SSMs) and convolutions, which scale sub-quadratically in context length. I describe the H3 (Hungry Hungry Hippos) architecture, a gated SSM architecture that matches Transformers in quality up to 3B parameters and achieves 2.4x faster inference. Second, I focus on developing hardware-aware algorithms for SSMs and convolutions. I describe FlashFFTConv, a fast algorithm for computing SSMs and convolutions on GPU by optimizing the Fast Fourier Transform (FFT). FlashFFTConv yields up to 7x speedup and 5x memory savings, even over vendor solutions from Nvidia. Third, I will briefly touch on how these same techniques can also be used to develop sub-quadratic scaling in the model dimension. I will describe Monarch Mixer, which uses a generalization of the FFT to achieve sub-quadratic scaling in both sequence length and model dimension. Throughout the talk, I will give examples of how these ideas are beginning to take hold, with gated SSMs and their variants now leading to state-of-the-art performance in long-context language models, embedding models, and DNA foundation models.Biography
Dan Fu is a PhD student in the Computer Science Department at Stanford University, where he is co-advised by Christopher Ré and Kayvon Fatahalian. His research interests are at the intersection of systems and machine learning. Recently, he has focused on developing algorithms and architectures to make machine learning more efficient, especially for enabling longer-context applications. His research has appeared as oral and spotlight presentations at NeurIPS, ICML, and ICLR, and he has received the best student paper runner up at UAI. Dan has also been supported by an NDSEG fellowship.Research Area/Interests
machine learning, systemsTIME Friday, April 19, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Apr24
EVENT DETAILS
less
Wednesday / CS Seminar
April 24th / 12:00 PM
In Person / Mudd 3514Speaker
R. James Cotton, Northwestern University & Shirley Ryan AbilityLabTalk Title
AI Powered Movement Analysis, Big Rehabilitation Data and a Path to Precision RehabilitationAbstract
This talk will discuss multiple methodological lines of work making movement and gait analysis more clinically accessible and biomechanically grounded. This includes reconstruction from synchronized multiview videos, smartphone videos, and wearable sensors. We will also discuss how implicit functions provide a powerful representation to map from time to joint angles, and GPU accelerated methods that enable end-to-end biomechanical fits from these different modalities. It will discuss some of the opportunities that large movement data enables, including the use of self-supervised learning to discover gait representations that can function as both diagnostic and response biomarkers. Finally, we will outline a vision for a Causal Framework for Precision Rehabilitation that can model this data to link from impairment to function and identify the optimal dynamic treatment policies to improve rehabilitation outcomes.Biography
I am an electrical engineer, neuroscientist, and physiatrist working as a physician-scientist at Shirley Ryan AbilityLab and Assistant Professor in the Northwestern University Department of Physical Medicine and Rehabilitation. I completed my residency in PM&R at Shirley Ryan AbilityLab (formers Rehabilitation Institute of Chicago) where I remained as faculty. Prior to that I obtained a B.S. in Electrical Engineering from Rice University followed by an MD and PhD in systems neuroscience from Baylor College of Medicine. My lab works at the intersection of artificial intelligence, wearable sensors, computer vision, causal and biomechanical modeling, and novel technologies to more precisely monitor and improve rehabilitation outcomes.Research Area/Interests
AI, computer vision, gait analysis, wearable sensors, rehabilitationTIME Wednesday, April 24, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Apr25
EVENT DETAILS
lessSpring is finally here. Enjoy the good vibes to come with some free bagels and coffee courtesy the Computer Science Department. Mix and mingle with fellow CS students and faculty every last Thursday of the month.
TIME Thursday, April 25, 2024 at 9:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Apr29
EVENT DETAILS
lessTBA
TIME Monday, April 29, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May1
EVENT DETAILS
lessTBA
TIME Wednesday, May 1, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May3
EVENT DETAILS
lessTBA
TIME Friday, May 3, 2024 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
May30
EVENT DETAILS
lessTBA
TIME Thursday, May 30, 2024 at 9:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
