Discussion
Book chapters are an important form of scholarly communication in the humanities and social sciences, yet have historically had very poor metadata coverage.
In a pilot study, I fine-tuned the #NuExtract3 vision-language model on a ground-truth dataset of 600+ table-of-contents scans from @DNB_Aktuelles's CC0-licensed "Kataloganreicherung" program. (1/3)
The ground-truth dataset is built by gating independent vision-LLM readings of the same TOC scan against each other for agreement, with a strong multimodal AI agent (Claude) arbitrating disagreements.
Status so far: 1,251 books sourced, 776 with ground truth, using Qwen3-Omni-30B-A3B-Instruct, Mistral-Small-3.2-24B-Instruct and NuExtract3 as readers.
Fine-tuning NuExtract3 (LoRA) on 624 train / 44 eval books from this pilot corpus markedly improved precision (0.61→0.76), with recall essentially unchanged (0.76→0.75) vs. the zero-shot base model. Next: scale up the training set.
Code: https://github.com/cboulanger/finetune-nuextract-toc
Try it: https://huggingface.co/spaces/cmboulanger/nuextract3-toc-demo (3/3)
The idea is to iteratively improve the pipeline and the model and become independent of frontier models eventually - NuExtract3 runs on a (good) laptop. #OpenScience #OpenWeightsModels
Install bonfire.cafe
Get the full app experience
