Just FYI: this is part of the damage being done by extractive AI companies. Their scraper bots routinely DDOS us, and they multiply to take up as much bandwidth as is available. We are having to put a huge percentage of our developers’ time into just keeping the site alive rather than building its future. They are actively destroying everything worthwhile about the internet. @hello https://hcommons.social/@hello/115854618561953712
Post
Replies:
6
@kfitz
Kathleen, Cloudflare offers a free technology specifically to prevent LLM scraping.
https://blog.cloudflare.com/cloudflare-ai-audit-control-ai-content-crawlers/
It was designed specifically to help with use cases like yours. I am a Sr. Solutions Engineer and can help set you up if you like.
@suzannealdrich @hello Hi, and thanks! We're already behind Cloudflare but still getting clobbered. I'll pass your offer to our dev team, though -- fine-tuning may help!
@kfitz @hello sadly, this comes from the top of the pyramid (as bad as the metaphor is). KC is based on Wordpress, and Wordpress leadership now actively promote AI as a foundational tool for the future of the web bla bla bla 😰 https://wordpress.org/news/2025/12/sotw-2025/
@kfitz @hello I hear you on this. It's been a total nightmare for Reclaim as well, what's worse is that blocking them is a joke, no one honors it. We had a site go from 1-2 million hits on avg per month to 7-8 million hits on avg basically overnight and it's all bots. Makes running a site with significant content and history exponentially more resource intensive.
