Post
As high-profile websites vanish, it’s a reminder that the web has no built-in archival layer.
But some publishers are now blocking the Wayback Machine.
What’s at stake if the web stops being archived? Our new FAQ explains: preserving the public record matters. 🌐📚 https://help.archive.org/help/faq-publishers-blocking-the-wayback-machine/
As i understand it, mostly this is a byproduct of sites trying to block AI scrapers. Would it be possible for the archive to publish a list of the IP addresses used by it's crawlers so that they can be explicitly whitelisted by sites that don't want to knock you out as collateral damage? Or is this already done, even? (If it is i'd appreciate a link to the list so i can whitelist you on my sites, unimportant though they be.)
@internetarchive such blockades of #InternetArchive should be outlawed!
Install bonfire.cafe
Get the full app experience
