Discussion
What if the rules meant to stop AI scraping also stopped us from saving the web?
The future of the open web is being debated this week in a conference room in London.
How can you help? Get informed ⤵️
https://cdt.org/insights/will-the-open-internet-survive-the-war-on-bots-mapping-the-debate-over-ai-preferences/
@internetarchive It’s so simple to scrape websites without abusing them, slowly and using the solution they provide for this operation.
As an example, ListenBrainz provides a file with all the website content for it to be downloaded and stored, but AI scrapers skip past that and keep scraping the website page-per-page, which slows the website down like a DDOS attack (as it happens periodically) and is part of the cause for ListenBrainz’s recent issues.
Install bonfire.cafe
Get the full app experience
