News & Events

Department Events & Announcements

Events

• Oct

  1

  IDEAL Theory of Deep Learning Special Quarter Seminar - Babak Hassibi, California Institute of Technology

  Department of Computer Science

  11:30 AM

  EVENT DETAILS

  IDEAL Theory of Deep Learning Special Quarter Seminar

  Title: The Blind Men and the Elephant: The Mysteries of Deep Learning

  Speaker: Babak Hassibi, Mose and Lillian S. Bohn Professor of Electrical Engineering and Computing and Mathematical Sciences, California Institute of Technology

  Time: Thursday October 1, 2020, 11:30-12:30

  Venue: See https://www.ideal.northwestern.edu/special-quarters/fall-2020/ for streaming link.

  Abstract: Deep learning has demonstrably enjoyed a great deal of recent practical success and is arguably the main driver behind the resurgent interest in machine learning and AI. Despite its tremendous empirical achievements, we are far from a theoretical understanding of deep networks. In this talk, we will argue that the success of deep learning is not only due to the special deep architecture of the models, but also due to the behavior of the stochastic descent methods used, which play a key role in reaching "good" solutions that generalize well to unseen data. We will connect learning algorithms such as stochastic gradient descent (SGD) and stochastic mirror descent (SMD) to work in H-infinity control in the 1990's, and thereby explain the convergence and implicit-regularization behavior of the algorithms when we are highly over-parametrized (what is now being called the "interpolating regime"). This gives us insight into why deep networks exhibit such powerful generalization abilities, a phenomenon now being referred to as "the blessing of dimensionality".

  Biography: Babak Hassibi was born in Tehran, Iran, in 1967. He received the B.S. in electrical engineering degree from the University of Tehran, Tehran, Iran, in 1989, and the M.S. and Ph.D. degrees in electrical engineering from Stanford University, Stanford, CA, USA, in 1993 and 1996, respectively. Since January 2001, he has been with the California Institute of Technology, where he is currently the Mose and Lilian S. Bohn Professor of electrical engineering. From 2008 to 2015, he was Executive Officer of electrical engineering, as well as Associate Director of information science and technology. From November 1998 to December 2000, he was a member of the Technical Staff with the Mathematical Sciences Research Center, Bell Laboratories, Murray Hill, NJ, USA. He has also held short-term appointments with the Ricoh California Research Center, the Indian Institute of Science, and Linkoping University, Sweden. His research interests include communications and information theory, control and network science, and signal processing and machine learning. He is the co-author of the books (both with A.H. Sayed and T. Kailath) Indefinite Quadratic Estimation and Control: A Unified Approach to $H^2$ and $H^\infty$ Theories (SIAM, 1999) and Linear Estimation (Prentice-Hall, 2000). He is a recipient of the Alborz Foundation Fellowship, 1999 O. Hugo Schuck Best Paper Award of the American Automatic Control Council (with H. Hindi and S. P. Boyd), 2002 National Science Foundation Career Award, 2002 Okawa Foundation Research Grant for Information and Telecommunications, 2003 David and Lucille Packard Fellowship for Science and Engineering, 2003 Presidential Early Career Award for Scientists and Engineers (PECASE), and 2009 Al-Marai Award for Innovative Research in Communications, and was a participant in the 2004 National Academy of Engineering “Frontiers in Engineering” program.

  more less

  TIME Thursday, October 1, 2020 at 11:30 AM - 12:30 PM

  ADD TO CALENDAR

  CONTACT Pam Villalovoz    pmv@northwestern.edu EMAIL

  CALENDAR Department of Computer Science

• Apr

  19

  CS Seminar: Hardware-Aware Efficient Primitives for Machine Learning (Dan Fu)

  Department of Computer Science (CS)

  12:00 PM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  

  Friday / CS Seminar
  April 19th / 12:00 PM
  In Person / Mudd 3514

  Speaker
  Dan Fu, Stanford University

  Talk Title
  Hardware-Aware Efficient Primitives for Machine Learning

  Abstract
  Efficiency is increasingly tied to quality to machine learning, with more efficient training algorithms leading to more powerful models. However, today's most popular machine learning models are built on asymptotically inefficient primitives. For example, attention in Transformers scales quadratically in the input size, while MLPs scale quadratically in model dimension. In this talk, I discuss my work on improving the efficiency of the core primitives in machine learning, with an emphasis on hardware-aware algorithms and long-context applications. First, I focus on replacing attention with gated state space models (SSMs) and convolutions, which scale sub-quadratically in context length. I describe the H3 (Hungry Hungry Hippos) architecture, a gated SSM architecture that matches Transformers in quality up to 3B parameters and achieves 2.4x faster inference. Second, I focus on developing hardware-aware algorithms for SSMs and convolutions. I describe FlashFFTConv, a fast algorithm for computing SSMs and convolutions on GPU by optimizing the Fast Fourier Transform (FFT). FlashFFTConv yields up to 7x speedup and 5x memory savings, even over vendor solutions from Nvidia. Third, I will briefly touch on how these same techniques can also be used to develop sub-quadratic scaling in the model dimension. I will describe Monarch Mixer, which uses a generalization of the FFT to achieve sub-quadratic scaling in both sequence length and model dimension. Throughout the talk, I will give examples of how these ideas are beginning to take hold, with gated SSMs and their variants now leading to state-of-the-art performance in long-context language models, embedding models, and DNA foundation models.

  Biography
  Dan Fu is a PhD student in the Computer Science Department at Stanford University, where he is co-advised by Christopher Ré and Kayvon Fatahalian. His research interests are at the intersection of systems and machine learning. Recently, he has focused on developing algorithms and architectures to make machine learning more efficient, especially for enabling longer-context applications. His research has appeared as oral and spotlight presentations at NeurIPS, ICML, and ICLR, and he has received the best student paper runner up at UAI. Dan has also been supported by an NDSEG fellowship.

  Research Area/Interests
  machine learning, systems

  more less

  TIME Friday, April 19, 2024 at 12:00 PM - 1:00 PM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• Apr

  24

  CS Seminar: AI Powered Movement Analysis, Big Rehabilitation Data and a Path to Precision Rehabilitation (R. James Cotton)

  Department of Computer Science (CS)

  12:00 PM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  

  Wednesday / CS Seminar
  April 24th / 12:00 PM
  In Person / Mudd 3514

  Speaker
  R. James Cotton, Northwestern University & Shirley Ryan AbilityLab

  Talk Title
  AI Powered Movement Analysis, Big Rehabilitation Data and a Path to Precision Rehabilitation

  Abstract
  This talk will discuss multiple methodological lines of work making movement and gait analysis more clinically accessible and biomechanically grounded. This includes reconstruction from synchronized multiview videos, smartphone videos, and wearable sensors. We will also discuss how implicit functions provide a powerful representation to map from time to joint angles, and GPU accelerated methods that enable end-to-end biomechanical fits from these different modalities. It will discuss some of the opportunities that large movement data enables, including the use of self-supervised learning to discover gait representations that can function as both diagnostic and response biomarkers. Finally, we will outline a vision for a Causal Framework for Precision Rehabilitation that can model this data to link from impairment to function and identify the optimal dynamic treatment policies to improve rehabilitation outcomes.

  Biography
  I am an electrical engineer, neuroscientist, and physiatrist working as a physician-scientist at Shirley Ryan AbilityLab and Assistant Professor in the Northwestern University Department of Physical Medicine and Rehabilitation. I completed my residency in PM&R at Shirley Ryan AbilityLab (formers Rehabilitation Institute of Chicago) where I remained as faculty. Prior to that I obtained a B.S. in Electrical Engineering from Rice University followed by an MD and PhD in systems neuroscience from Baylor College of Medicine. My lab works at the intersection of artificial intelligence, wearable sensors, computer vision, causal and biomechanical modeling, and novel technologies to more precisely monitor and improve rehabilitation outcomes.

  Research Area/Interests
  AI, computer vision, gait analysis, wearable sensors, rehabilitation

  more less

  TIME Wednesday, April 24, 2024 at 12:00 PM - 1:00 PM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• Apr

  25

  TGI Bagel Thursday

  Department of Computer Science (CS)

  9:00 AM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  Spring is finally here. Enjoy the good vibes to come with some free bagels and coffee courtesy the Computer Science Department. Mix and mingle with fellow CS students and faculty every last Thursday of the month.

  more less

  TIME Thursday, April 25, 2024 at 9:00 AM - 11:00 AM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• Apr

  29

  CS Seminar: Kaize Ding

  Department of Computer Science (CS)

  12:00 PM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  TBA

  more less

  TIME Monday, April 29, 2024 at 12:00 PM - 1:00 PM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• May

  1

  CS Seminar: Tong Zhang

  Department of Computer Science (CS)

  12:00 PM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  TBA

  more less

  TIME Wednesday, May 1, 2024 at 12:00 PM - 1:00 PM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• May

  3

  CS Seminar: Shafi Goldwasser

  Department of Computer Science (CS)

  12:00 PM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  TBA

  more less

  TIME Friday, May 3, 2024 at 12:00 PM - 1:00 PM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• May

  30

  TGI Bagel Thursday

  Department of Computer Science (CS)

  9:00 AM 3514, Mudd Hall ( formerly Seeley G. Mudd Library)

  EVENT DETAILS

  TBA

  more less

  TIME Thursday, May 30, 2024 at 9:00 AM - 11:00 AM

  LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library)    map it

  ADD TO CALENDAR

  CONTACT Wynante R Charles    wynante.charles@northwestern.edu EMAIL

  CALENDAR Department of Computer Science (CS)

• Jun

  10

  Northwestern Engineering PhD Hooding and Master's Degree Recognition Ceremony

  McCormick School of Engineering and Applied Science

  9:00 AM Welsh-Ryan Arena

  EVENT DETAILSmore info

  McCormick School of Engineering PhD Hooding and Master’s Degree Recognition Ceremony

  more less

  TIME Monday, June 10, 2024 at 9:00 AM - 11:00 AM

  LOCATION Welsh-Ryan Arena   

  ADD TO CALENDAR

  CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

  CALENDAR McCormick School of Engineering and Applied Science

• Jun

  10

  Northwestern Engineering Undergraduate Convocation

  McCormick School of Engineering and Applied Science

  2:00 PM Welsh-Ryan Arena

  EVENT DETAILSmore info

  McCormick School of Engineering Undergraduate Convocation

  more less

  TIME Monday, June 10, 2024 at 2:00 PM - 4:00 PM

  LOCATION Welsh-Ryan Arena   

  ADD TO CALENDAR

  CONTACT Amy Pokrass    amy.pokrass@northwestern.edu EMAIL

  CALENDAR McCormick School of Engineering and Applied Science