Post · bonfire.cafe

@ElenLeFoll@fediscience.org · 2 weeks ago

Our final keynote speaker is also a doctoral researcher, Sam Boeve, presenting "From Tool to Theory: LLMs in Psycholinguistics". Sam presented three case studies, the last of which is based on his recent work on Dutch LLM's surprisal estimates in sentence, paragraph and book reading (https://doi.org/10.3758/s13428-025-02774-4). As previous research with English has shown, it turns out that not one size fits all and larger models are not necessarily better. 8/🧵

#LLM #linguistics #psycholinguistics

A systematic evaluation of Dutch large language models’ surprisal estimates in sentence, paragraph and book reading

Sam presenting a slide that reads: Best Dutch model
• Dutch specific › Multilingual
• Less parameters (smaller models)
work better
• Overall:
gpt2-small-dutch gpt2-medium-dutch-embeddings (de Vries & Nissim,
2020) — Sam presenting a slide that reads: Best Dutch model • Dutch specific › Multilingual • Less parameters (smaller models) work better • Overall: gpt2-small-dutch gpt2-medium-dutch-embeddings (de Vries & Nissim, 2020)

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

But what do we mean by LLMs exactly? To ensure that everyone is on the same page, co-organiser @nilsreiter is giving a pre-workshop lecture "An Introduction to Large Language Models: LLMs 101". Attendees get introduced to important distinctions such as user prompt vs. system prompt, pre-training vs. fine-tuning, and commercial vs. locally-run open-source models. 2/🧵

#LLMs #GenAI #linguistics #research

Nils at the lectern presenting a slide entitled LLMs and products. It shows a diagram for research whereby a prompt is entered in a model and there is an output and a second diagram for commercial application whereby a prompt is fed into a question mark, then entered in a model, then fed through another question mark, and then there is an output.

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

Our first keynote speaker is @tschfflr from the Ruhr University Bochum. Tatjana's talk is entitled "Linguistic analysis with LLMs? The case of discourse and pragmatics" and she's brought along no fewer than three studies from her team's work on metaphor detection in religious language, the semantics and pragmatics of emojis, and the detection of hate and candy speech. Her implications for #linguistics research are both very cautious and optimistic as to the future of linguistics. 3/🧵

#LLM #GenAI #metaphor

Tatjana presenting a slide on annotation, data and corpora.

Tatjana speaking at the lectern with an opening slide featuring the image of a robot writing with a feather

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

Our second keynote speaker of the day is a Mercator Fellow of the CRC "Prominence in Language" @dingemansemark. Mark's asks "What makes LLMs so irresistible?" and begins with our ancient "amazing ability to make sense of nonsense" citing bone-throwing, hand-reading, and horoscopes, before moving to ELIZA and ChatGPT. Mark stresses the need to distinguish between 'understanding' and 'regurgitating', and 'predicting' and 'responding' and introduces what he sees as useful mental models to describe LLMs such as 'stochastic parrots' and 'mansplaining as a service'. In the second half, Mark walked us through the many problems he sees with a recent publication on the supposed metalinguistic abilities of LLMs. Much food for thought for the discussion over coffee and the poster session! 4/🧵

#LLM #GenAI #linguistics

Mark presenting a slide with a quote:
"Decisions made by the general public about emergent technologies depend much more on what that public attributes to such technologies than on what they actually are or can and cannot do"
Joseph Weizenbaum
Computer Power and Human Reason
1976 — Mark presenting a slide with a quote: "Decisions made by the general public about emergent technologies depend much more on what that public attributes to such technologies than on what they actually are or can and cannot do" Joseph Weizenbaum Computer Power and Human Reason 1976

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

It was a busy and lively poster session with presenters ranging from M.A. students to professors (check out the full list: https://sfb1252.github.io/llm-workshop/#poster-presentations), lots of discussions on the use of #LLMs for #linguistics research fulled by copious quantities of coffee and tray-fulls of cup cakes! We now take a short break before continuing the conversations over dinner. 5/🧵

#LLM #linguistics

Two people looking at poster, whilst holding a cup of coffee. Cup cakes in the background.

A person presenting a poster and two people listening.

Tatjana and Jürgen in front of poster about "Simulating Human Judgments
The Potential of LLMs in Linguistic Experiments on Prominence" — Tatjana and Jürgen in front of poster about "Simulating Human Judgments The Potential of LLMs in Linguistic Experiments on Prominence"

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

Oups, I nearly forgot to present our own poster... Luckily, Tiziana Ilie, M.A. student and first author on this project, did a grand job presenting our work on how DeepL deals with pronomial pickups of the hybrid noun "Mädchen". For this study, we used the #OpenSource model OLMo-2-7B to generate our experimental stimuli. Our abstract is here: https://paths.to/m%C3%A4dchen-poster and we will have a preprint out soon so watch this space...

#MachineTranslation #LLM #linguistics

Tiziana and I presenting our poster about "Genus-sexus congruence in machine translation: DeepL's pronominal resumption of 'das Mädchen'"

Elen Le Foll 🇫🇷 🇬🇧 🇩🇪

@ElenLeFoll@fediscience.org replied · 2 weeks ago

We began day 2 of our Large Language Models (LLM) for linguistics research workshop @UniKoeln with a fascinating keynote by Charlotte Pouw on "Interpreting models for speech generation and understanding using methods from #psycholinguistics". Charlotte shared insights from two studies from her #PhD research. Check out the papers: https://aclanthology.org/2025.conll-1.9/ and https://direct.mit.edu/coli/article/50/4/1557/123790. 7/🧵

#LLM #NLP #linguistics

Charlotte presenting a slide with plots entitled The Role of Data Exposure

1 more replies (not shown)

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances

Bonfire social · 1.0.1-alpha.8 no JS en

Automatic federation enabled