NLP@U period-3 meeting

The 2025-2026 period-3 meeting of NLP@U will be hosted by the Faculty of Social and Behavioural Sciences.

Below you will find information about the speakers, titles and abstracts. After the talks, there will be a round of short pitches (see information below), and lunch. In addition, there will be a coffee break and plenty of time to network and chat informally.

Title and abstract

Speaker

Walter Daelemans (University of Antwerp).

Title

How Stable Are the Political Preferences Expressed by LLMs?

Abstract

Voting advice applications are often launched before elections. Such applications are built around a curated set of statements designed to be both representative and discriminative across political parties. Users indicate whether they agree or disagree with each statement, thereby revealing their social and economic preferences. By comparing these responses with the positions of political parties on the same statements, the application can suggest which party is the best match.

To investigate political bias in LLMs, we use Dutch-language voting tests developed by political scientists Stefaan Walgrave and Michiel Nuytemans for previous Belgian elections.

We asked three sizes of OpenAI models, as well as flagship models from Anthropic, Google, and xAI, to respond to a balanced subset of these statements. We then compare the models along left-right and socio-economic dimensions. We replicate earlier findings showing that commercial models, including Grok, tend to lean left or progressive. We then repeat the experiment with two perturbations: translating the Dutch statements into English and reversing their polarity, and two prompt manipulations: assigning the models a persona and informing them about the user’s political preferences. We identify which models and which statements are most sensitive to these changes.

We interpret the results in the context of earlier work on sycophancy and prompt sensitivity.

Speaker

Ine Gevers (University of Antwerp).

Title

When LLMs Miss the Hint: Evaluating Abductive Reasoning in LLMs Through Language Games

Abstract

Large language models (LLMs) have achieved striking successes on many benchmarks, yet recent studies continue to expose fundamental weaknesses. We introduce Concept, a simple word-guessing board game, as a benchmark for probing abductive reasoning. Our results show that this game, easily solved by humans (with a success rate of over 90%), is still very challenging for state-of-the-art LLMs (no model exceeds 40% success rate). Specifically, we observe that LLMs struggle with interpreting other players’ strategic intents, and with correcting initial hypotheses given sequential information updates. In addition, we extend the evaluation across multiple languages, and find that the LLM performance drops further in lower-resource languages (Dutch, French, and Spanish) compared to English. 

Speaker

Verna Dankers (Mila & McGill University).

Title

Understanding the Memorization vs Generalization Balance in LLMs

Abstract

In this talk, I focus on a memory injection paradigm and present two use cases. Firstly, I introduce our new unlearning testbed, LACUNA, which injects synthetic personally identifiable information (PII) into known parameters to evaluate whether unlearning methods actually target the parameters that store the memories they aim to unlearn. We use LACUNA to demonstrate that state-of-the-art unlearning methods do not target the correct parameters, raising questions about whether they truly unlearn or merely obfuscate PII. Secondly, I present Tiered Language Models, in which the memory injection is aimed at safeguarding certain information and capabilities. These can only be unlocked with the right key, enabling controlled access in open-weight models.

Pitches and discussion

We are inviting submissions for the pitches-and-discussion section. Participants may submit a tentative title and a description. Each selected participant will have two (2) minutes to either:

1. Describe an idea you are working on, and ask if anyone is interested in discussing this idea with you, or
2. Propose a topic for debate immediately afterwards.

Each pitch will be followed by 5 minutes of discussion.

Please fill out this form to submit your pitch.

Programme