Events
LMSS @ Cornell Tech: Eunsol Choi (UT Austin)
Knowledge Augmentation for Language Models
Modern language models (LMs) store and use immense amounts of knowledge about the real world. However, their knowledge, acquired during pre-training on web corpora, can be incorrect, misleading or outdated. In this talk, I will discuss two complementary avenues for augmenting knowledge in LMs. First, I will present a modular, retrieval-based approach which provides new information in-context at inference time. We improve in-context retrieval augmentation by learning a compressor which summarizes retrieved documents into textual summaries, enabling adaptive and efficient augmentation. In the second half of the talk, I will present a parameter updating approach which aims to enable models to internalize new information. Our distillation-based approach outperforms existing approaches in propagating injected knowledge to enable broader inferences. Together, this talk outlines the challenges and progresses of knowledge augmentation for LMs.
Pizza will be served at 12 p.m.
This event is part of the Learning Machines Seminar Series at Cornell Tech. The seminar focuses on machine learning and related areas, including Natural Language Processing, Vision, and Robotics. The series is organized by Associate Professor Yoav Artzi and sponsored by Bloomberg.
Speaker Bio
Eunsol Choi is an assistant professor in the Computer Science department at the University of Texas at Austin. Prior to UT, she spent a year at Google AI as a visiting researcher. Her research area spans natural language processing and machine learning. She is particularly interested in interpreting and reasoning about text in a dynamic real world context. She received a Ph.D. from University of Washington and B.A from Cornell University. She is a recipient of a Facebook research fellowship, Google faculty research award, Sony faculty award, and an outstanding paper award at EMNLP.