Introduction to In-Context Learning

We cordially invite you to our Introduction to In-Context Learning workshop organized by Saarland University. The workshop will be held online on June 12th from 2 pm to 5 pm.

In-context learning (ICL) is a popular inference strategy where large language models solve a task using only a few labeled demonstrations without needing any parameter updates. This technique can help people from all backgrounds take advantage of the capabilities of LLMs without requiring much training or knowledge on their part. During this workshop, we will do a brief introduction on the topic focusing on a multilingual approach. We will utilize several NLP tasks to analyze the importance of the quality of the demonstrations, customize templates, and other advanced applications of ICL.

Research talk

by Miaoran Zhang

The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis

In-context learning is a popular inference strategy where large language models solve a task using only a few labelled demonstrations without needing any parameter updates. Compared to work on monolingual (English) in-context learning, multilingual in-context learning is under-explored, and we lack an in-depth understanding of the role of demonstrations in this context. To address this gap, we conducted a multidimensional analysis of multilingual in-context learning, experimenting with 5 models from different model families, 9 datasets covering classification and generation tasks, and 56 typologically diverse languages. Our results reveal that the effectiveness of demonstrations varies significantly across models, tasks, and languages. We also find that Llama 2-Chat, GPT-3.5, and GPT-4 are largely insensitive to the quality of demonstrations. Instead, a carefully crafted template often eliminates the benefits of demonstrations for some tasks and languages altogether. These findings show that the importance of demonstrations might be overestimated. Our work highlights the need for granular evaluation across multiple axes towards a better understanding of in-context learning.

Practical Session

by Paloma García de Herreros García and Israel A. Azime

During the practical section of this workshop, we will cover the following topics:

Basic In-Context Learning Demonstration
The Impact of Demonstration Quality
Customizing Templates for In-Context Learning
Multilingual In-Context Learning Exploration, Functions, Tools and Agents.
Advanced Applications of In-Context Learning, chain-of-thought prompting and Agents.