Tutorials

Abstract: The ‘symbol grounding’ problem refers to the ability of models to establish non-arbitrary links between symbols (for example, words or phrases) and data from perception and experience. In recent years, several deep, transformer-based models have been proposed which purport to learn such relationships, following extensive pretraining on datasets consisting of images (or videos) paired with text. One of the goals of this lecture will be to provide an overview of the state of the art in the field, with particular reference to available datasets consisting of images and text, model architectures, and tasks. One of the key questions that arises here is: What do these models learn to ground? We will see that there is a strong object-centric bias in much of the training data used for such models, and will address some research to overcome this bias. We then turn to some recent research focusing on model analysis and evaluation, delving into recent work on analysing the grounding capabilities of such models, using a variety of techniques, including foil-based methods and ablation.

Bio: Albert Gatt is Associate Professor in the Natural Language Processing Group at the Department of Information and Computing Sciences, Utrecht University. He is also affiliated with the Institute of Linguistics and Language Technology, University of Malta. His main research interests are in Natural Language Generation and the interface between Vision and Language. He has also worked extensively on NLP for under-resourced languages.

Abstract: Deep neural models of language perform very well in many tasks but are hard to understand and interpret. Various families of techniques and methodologies have been developed in recent years to better understand the inner-dynamics of these models and the nature of linguistic knowledge they encode. In this tutorial, we look at the most common approaches, ranging from hypothesis-driven methods for probing the internal representations of deep models, to data-driven methods for tracing and analyzing information flow in deep models and quantifying its impact on model performance and outcome.

Bio: Afra Alishahi is an Associate Professor of Cognitive Science and Artificial Intelligence at Tilburg University, the Netherlands. Her main research interests are developing computational models for studying the process of human language acquisition, studying the emergence of linguistic structure in grounded models of language learning, and developing tools and techniques for analyzing linguistic representations in neural models of language. She is a co-recipient of a consortium grant from Dutch National Research Agenda on “Interpreting Deep Learning Models for Text and Sound”. She is a co-founder of the BlackboxNLP workshops, an official venue dedicated to analyzing and interpreting neural networks for NLP.

Short Abstract: Deep Learning (DL) has pushed the frontier in Natural Language Processing (NLP), but DL typically requires abundance of training data with high-quality labels. However, in many situations data is scarce and not of the right kind. In this tutorial, we will revisit approaches to transfer learning including multi-task learning (MTL) to tackle the problem of data scarcity and label bias. I will revisit how transfer and MTL has evolved over the last half decade and discuss example cases.

Short Bio: Barbara Plank is Chair (Full Professor) for AI and Computational Linguistics at LMU Munich, co-director of the Center for Information and Language Processing (CIS) and lead of the MaiNLP research lab. Besides LMU, she holds a secondary part-time appointment at the Computer Science Department of the IT University of Copenhagen, co-leading the NLP North research unit.

Abstract: Large neural network models appear to produce impressive performance on a range of language understanding and generation tasks, but are notoriously referred to as “black boxes”. Without an understanding of how such models work under the hood–i.e., what representations and algorithms underlie their linguistic reasoning–it is difficult to translate their success into new theories or insights about language itself, i.e., with relevance to linguistics and cognitive science. This talk will focus on methods for looking “under the hood” and how large neural networks structure information internally, and how they “reason” about problems, focusing on important issues in linguistics such as compositionality, systematicity, and productivity.

Bio: Ellie is an Assistant Professor of Computer Science at Brown University, and a Research Scientist at Google. She is interested in understanding how language models (especially large, neural networks models) work under the hood, and in characterizing how the representations and algorithms they use reflect and differ from those of humans.