News & EventsDepartment Events & Announcements
Events
-
Feb8
EVENT DETAILS
lessWednesday / CS Seminar
February 8th / 10:00 AM
Hybrid / Mudd 3514Title: Controlling Large Language Models: Generating (Useful) Text from Models We Don’t Fully Understand
Speaker: Ari HoltzmanAbstract:
Generative language models have recently exploded in popularity, with services such as ChatGPT deployed to millions of users. These neural models are fascinating, useful, and incredibly mysterious: rather than designing what we want them to do, we nudge them in the right direction and must discover what they are capable of. But how can we rely on such inscrutable systems?This talk will describe a number of key characteristics we want from generative models of text, such as coherence and correctness, and show how we can design algorithms to more reliably generate text with these properties. We will also highlight some of the challenges of using such models, including the need to discover and name new and often unexpected emergent behavior. Finally, we will discuss the implications this has for the grand challenge of understanding models at a level where we can safely control their behavior.
Biography:
Ari Holtzman is a PhD student at the University of Washington. His research has focused broadly on generative models of text: how we can use them and how can we understand them better. His research interests have spanned everything from dialogue, including winning the first Amazon Alexa Prize in 2017, to fundamental research on text generation, such as proposing Nucleus Sampling, a decoding algorithm used broadly in deployed systems such as the GPT-3 API and academic research. Ari completed an interdisciplinary degree at NYU combining Computer Science and the Philosophy of Language.TIME Wednesday, February 8, 2023 at 10:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb8
EVENT DETAILS
lessWednesday / CS Seminar
February 8th / 12:00 PM
Mudd 3514Title: Cryptography, Security, and Law
Speaker: Sunoo ParkAbstract:
My research focuses on the security, privacy, and transparency of technologies in societal and legal context. My talk will focus on three of my recent works in this space, relating to (1) preventing exploitation of stolen email data, (2) enhancing accountability in electronic surveillance, and (3) legal risks faced by security researchers.Biography:
Sunoo Park is a Postdoctoral Fellow at Columbia University and Visiting Fellow at Columbia Law School. Her research interests range across cryptography, security, and technology law. She received her Ph.D. in computer science at MIT, her J.D. at Harvard Law School, and her B.A. in computer science at the University of Cambridge. She has also been affiliated with Cornell Tech's Digital Life Initiative, the Berkman Klein Center for Internet & Society at Harvard University, the MIT Media Lab's Digital Currency Initiative, and MIT's Internet Policy Research Initiative.TIME Wednesday, February 8, 2023 at 12:00 PM - 1:00 PM
LOCATION Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb13
EVENT DETAILS
lessWednesday / CS Seminar
February 13th / 10:00 AM
Mudd 3514Title: Distance-Estimation in Modern Graphs: Algorithms and Impossibility
Speaker: Nicole WeinAbstract:
The size and complexity of today's graphs present challenges that necessitate the discovery of new algorithms. One central area of research in this endeavor is computing and estimating distances in graphs. In this talk I will discuss two fundamental families of distance problems in the context of modern graphs: Diameter/Radius/Eccentricities and Hopsets/Shortcut Sets.The best known algorithm for computing the diameter (largest distance) of a graph is the naive algorithm of computing all-pairs shortest paths and returning the largest distance. Unfortunately, this can be prohibitively slow for massive graphs. Thus, it is important to understand how fast and how accurately the diameter of a graph can be approximated. I will present tight bounds for this problem via conditional lower bounds from fine-grained complexity.
Secondly, for a number of settings relevant to modern graphs (e.g. parallel algorithms, streaming algorithms, dynamic algorithms), distance computation is more efficient when the input graph has low hop-diameter. Thus, a useful preprocessing step is to add a set of edges (a hopset) to the graph that reduces the hop-diameter of the graph, while preserving important distance information. I will present progress on upper and lower bounds for hopsets.
Biography:
Nicole Wein is a Simons Postdoctoral Leader at DIMACS at Rutgers University. Previously, she obtained her Ph.D. from MIT advised by Virginia Vassilevska Williams. She is a theoretical computer scientist and her research interests include graph algorithms and lower bounds including in the areas of distance-estimation algorithms, dynamic algorithms, and fine-grained complexity.TIME Monday, February 13, 2023 at 10:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb13
EVENT DETAILS
lessWednesday / CS Seminar
February 13th / 12:00 PM
Mudd 3514Title: Reading to Learn: Improving Generalization by Learning From Language
Speaker: Victor ZhongAbstract:
Traditional machine learning systems are trained on vast quantities of annotated data or experience. These systems often do not generalize to new, related problems that emerge after training, such as conversing about new topics or interacting with new environments. In this talk, I present Reading to Learn, a new class of algorithms that improve generalization by learning to read language specifications, without requiring any actual experience or labeled examples. This includes, for example, reading FAQ documents to learn to answer questions about new topics and reading manuals to learn to play new games. I will discuss new algorithms and data for Reading to Learn applied to a broad range of tasks, including policy learning in grounded environments and data synthesis for code generation, while also highlighting open challenges for this line of work. Ultimately, the goal of Reading to Learn is to democratize AI by making it accessible for low-resource problems where the practitioner cannot obtain annotated data at scale, but can instead write language specifications that models read to generalize.Biography:
Victor Zhong is a PhD student at the University of Washington Natural Language Processing group. His research is at the intersection of natural language processing and machine learning, with an emphasis on how to use language understanding to learn more generally and more efficiently. His research covers a range of topics, including dialogue, code generation, question answering, and grounded reinforcement learning. Victor has been awarded the Apple AI/ML Fellowship as well as an EMNLP Outstanding Paper award. His work has been featured in Wired, MIT Technology Review, TechCrunch, VentureBeat, Fast Company, and Quanta Magazine. He was a founding member of Salesforce Research, and has previously worked at Meta AI Research and Google Brain. He obtained a Masters in Computer Science from Stanford University and a Bachelor of Applied Science in Computer Engineering from the University of Toronto.TIME Monday, February 13, 2023 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb15
EVENT DETAILS
lessWednesday / CS Seminar
February 15th / 10:00 AM
Mudd 3514Title: Computational Imaging for Enabling Vision Beyond Human Perception
Speaker: Mark SheininAbstract:
From minute surface vibrations to very fast-occurring events, the world is rich with phenomena humans cannot perceive. Likewise, most computer vision systems are primarily based on 'conventional' cameras, which were designed to mimic the imaging principle of the human eye, and therefore are equally blind to these ubiquitous phenomena. In this talk, I will show that we can capture these hidden phenomena by creatively building novel vision systems composed of common off-the-shelf components (i.e., cameras and optics) coupled with cutting-edge algorithms. Specifically, I will cover three projects using computational imaging to sense hidden phenomena. First, I will describe the ACam - a camera designed to capture the minute flicker of electric lights ubiquitous in our modern environments. I will show that bulb flicker is a powerful visual cue that enables various applications like scene light source unmixing, reflection separation, and remote analyses of the electric grid itself. Second, I will describe Diffraction Line Imaging, a novel imaging principle that exploits diffractive optics to capture sparse 2D scenes with 1D (line) sensors. The method's applications include capturing fast motions (e.g., actors and particles within a fast-flowing liquid) and structured light 3D scanning with line illumination and line sensing. Lastly, I will present a new approach for sensing minute high-frequency surface vibrations (up to 63kHz) for multiple scene sources simultaneously, using "slow" sensors rated for only 130Hz. Applications include capturing vibration caused by audio sources (e.g., speakers, human voice, and musical instruments) and localizing vibration sources (e.g., the position of a knock on the door).Biography:
Mark Sheinin is a Post-doctoral Research Associate at Carnegie Mellon University's Robotic Institute at the Illumination and Imaging Laboratory. He received his Ph.D. in Electrical Engineering from the Technion - Israel Institute of Technology in 2019. His work has received the Best Student Paper Award at CVPR 2017 and the Best Paper Honorable Mention Award at CVPR 2022. He received the Porat Award for Outstanding Graduate Students, the Jacobs-Qualcomm Fellowship in 2017, and the Jacobs Distinguished Publication Award in 2018. His research interests include computational photography and computer vision.TIME Wednesday, February 15, 2023 at 10:00 AM - 11:00 AM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb15
EVENT DETAILS
less
Join us for a series of professional development events hosted by McCormick HR. The first event of the series will focus on conflict resolution.
Conflict is an unavoidable part of life, both at home and at work. Knowing how to resolve conflict – and, in many cases, reap the benefits that conflict can bring – is a valuable skill. Participants in this workshop will learn how to iron out differences before they escalate, they will explore the dynamics of conflict, develop awareness of their role in conflict situations, and acquire tips for dealing with hostile individuals.
This event is voluntary and open to all Northwestern Engineering staff, faculty, postdocs, and research staff.
Training provided by SupportLinc, your Employee Assistance Program
TIME Wednesday, February 15, 2023 at 12:00 PM - 1:00 PM
LOCATION The Hive, Room 2-350, Ford Motor Company Engineering Design Center map it
CONTACT Kimberly Higgins kimberly.higgins@northwestern.edu EMAIL
CALENDAR McCormick School of Engineering and Applied Science
-
Feb15
EVENT DETAILS
lessWednesday / CS Seminar
February 15th / 12:00 PM
Mudd 3514Title: Democratizing Large-Scale AI Model Training via Heterogeneous Memory
Speaker: Dong LiAbstract:
The size of large artificial intelligence (AI) models increases by about 200x in the past three years. To train those models with billion- or even trillion-scale parameters, memory capacity becomes a major bottleneck, which leads to a range of functional and performance issues. The memory capacity problem becomes even worse with growth of batch size, data modality, and training pipeline size and complexity. Recent advance of heterogeneous memory (HM) provides a cost-effective approach to increase memory capacity. Using CPU memory as an extension to GPU memory, we can build an HM to enable large-scale AI model training without using extra GPUs to accommodate large memory consumption. However, not only HM imposes challenges on tensor allocation and migration on HM itself, but it is also unclear how HM affects training throughput. AI model training possesses unique characteristics of memory access patterns and data structures, which places challenges on the promptness of data migration, load balancing, and tensor redundancy on GPU. In this talk, we present our recent work on using HM to enable large-scale AI model training. We identify the major memory capacity bottleneck in tensors, and minimize GPU memory usage through co-offloading of computing and tensors from GPU. We also use analytical performance modeling to guide tensor migration between memory components in HM, in order to minimize migration volume and reduce load imbalance between batches. We show that using HM we can train industry-quality transformer models with over 13 billion parameters on a single GPU, a 10x increase in size compared to popular frameworks such as PyTorch, and we do so without requiring any model change from data scientists or sacrificing computational efficiency. Our work has been integrated into Microsoft DeepSpeed and employed in industry to democratize large-scale AI models. We also show that using HM we enable large-scale GNN training with billion-scale graphs without losing accuracy and suffering from out of memory (OOM).Biography:
Dong Li is an associate professor at EECS, University of California, Merced. Previously, he was a research scientist at the Oak Ridge National Laboratory (ORNL). Dong earned his PhD in computer science from Virginia Tech. His research focuses on high performance computing (HPC), and maintains a strong relevance to computer systems. The core theme of his research is to study how to enable scalable and efficient execution of enterprise and scientific applications (including large-scale AI models) on increasingly complex parallel systems. Dong received an ORNL/CSMD Distinguished Contributor Award in 2013, a CAREER Award from the National Science Foundation in 2016, a Berkeley Lab University Faculty Fellowship in 2016, a Facebook research award in 2021, and an Oracle research award in 2022. His paper in SC'14 was nominated as the best student paper. His paper in ASPLOS'21 won the distinguished artifact award. He was also the lead PI for the NVIDIA CUDA Research Center at UC Merced. He is an associate editor for IEEE Transactions on Parallel and Distributed Systems (TPDS).TIME Wednesday, February 15, 2023 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Feb27
EVENT DETAILS
lessMonday / CS Seminar
February 27th / 12:00 PM
Mudd 3514Title: Algorithms and Systems for Efficient Machine Learning
Speaker: Tri DaoAbstract:
Machine learning (ML) models training will continue to grow to consume more cycles, their inference will proliferate on more kinds of devices, and their capabilities will be used on more domains. Some goals central to this future are to make ML models efficient so they remain practical to train and deploy, and to unlock new application domains with new capabilities. We describe some recent developments in hardware-aware algorithms to improve the efficiency-quality tradeoff of ML models and equip them with long context. In the first half, we focus on structured sparsity, a natural approach to mitigate the extensive compute and memory cost of large ML models. We describe a line of work on learnable fast transforms which, thanks to their expressiveness and efficiency, yields some of the first sparse training methods to speed up large models in wall-clock time (2x) without compromising their quality. In the second half, we focus on efficient Transformer training and inference for long sequences. We describe FlashAttention, a fast and memory-efficient algorithm to compute attention with no approximation. By careful accounting of reads/writes between different levels of memory hierarchy, FlashAttention is 2-4x faster and uses 10-20x less memory compared to the best existing attention implementations, allowing us to train higher-quality Transformers with 8x longer context. FlashAttention is now widely used in some of the largest research labs and companies, in just 6 months after its release. We conclude with some exciting directions in ML and systems, such as software-hardware co-design, structured sparsity for scientific AI, and long context for new AI workflows and modalities.Biography:
Tri Dao is a PhD student in Computer Science at Stanford, co-advised by Christopher Ré and Stefano Ermon. He works at the interface of machine learning and systems, and his research interests include sequence models with long-range memory and structured matrices for compact deep learning models. His work has received the ICML 2022 Outstanding paper runner-up award.TIME Monday, February 27, 2023 at 12:00 PM - 1:00 PM
LOCATION 3514, Mudd Hall ( formerly Seeley G. Mudd Library) map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Mar11
EVENT DETAILS
lessWinter Classes End
TIME Saturday, March 11, 2023
CONTACT Office of the Registrar nu-registrar@northwestern.edu EMAIL
CALENDAR University Academic Calendar
-
Mar18
EVENT DETAILS
lessSpring Break Begins
TIME Saturday, March 18, 2023
CONTACT Office of the Registrar nu-registrar@northwestern.edu EMAIL
CALENDAR University Academic Calendar
-
Mar24
EVENT DETAILS
lessWinter Degrees Conferred
TIME Friday, March 24, 2023
CONTACT Office of the Registrar nu-registrar@northwestern.edu EMAIL
CALENDAR University Academic Calendar
-
Mar27
EVENT DETAILS
lessSpring Break Ends
TIME Monday, March 27, 2023
CONTACT Office of the Registrar nu-registrar@northwestern.edu EMAIL
CALENDAR University Academic Calendar
-
Mar28
EVENT DETAILS
lessSpring Classes begin 8 a.m. (Northwestern Monday: Classes scheduled to meet on Mondays meet on this day)
TIME Tuesday, March 28, 2023
CONTACT Office of the Registrar nu-registrar@northwestern.edu EMAIL
CALENDAR University Academic Calendar