“AI” crawlers do not follow the rules:
https://www.heise.de/en/background/Obituary-Farewell-to-robots-txt-1994-2025-10766991.html?seite=all

In order to keep our academic search engine https://www.base-search.net online, we were forced to lock out all browsers that cannot execute Javascript. We currently use #Anubis for this: https://anubis.techaro.lol/

We are sorry for the delay this causes for our legitimate users. Here is what you can do: Accept and keep our cookies! We do not use them for tracking. They help us remember that you are probably a human being.