@kkarhan @Erpel Hi! We have opted out of certain crawlers (see our robots.txt file) and routinely share our experiences with blocking aggressive scrapers (or the consequences of said scrapers on the reliability and availability of our platform).
If you believe we're missing any, our configurations are hosted here (but I can dig up the exact file for pull requests if you need me to): https://codeberg.org/Codeberg-Infrastructure
~n
@Codeberg my problem ain't #scrapers - otherwise I'd #SelfHost my stuff in my own home LAN - but rather #bots flooding #Issues and #PullRequests with garbage.
The problem I dread is once people start abusing their " #AI" #bullshit and #FloodTheZoneWithShit aka. " #AIslop" for no good reason.
And @Erpel 's original issue is just that: #spam in the #IssueTracker...
@kkarhan @Erpel @bagder Sorry, I misunderstood (due to the earlier mention of Copilot). As you probably know, spam/trolling is also still a problem for any public-facing service with open registrations. We can't guarantee that we can 100% prevent it either, unfortunately. (Although we'd like to believe we're getting better/faster at removing these on behalf of other people.)
Completely understand if you'd rather take the quieter route of self-hosting though. 😁 ~n
