Post · bonfire.cafe

Post

Log in

@h4ckernews@mastodon.social · 5 days ago

Facebook's Fascination with My Robots.txt

https://blog.nytsoi.net/2026/02/23/facebook-robots-txt

#HackerNews #Facebook #RobotsTxt #SocialMedia #TechNews #WebCrawlers

Facebook's Fascination with My Robots.txt

Facebook is requesting my robots.txt thousands of times per hour.

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances

Bonfire social · 1.0.2-alpha.34 no JS en

Automatic federation enabled

Log in