I've been playing "whack a mole" to suppress crawlers that violate robots.txt. It's pretty much hopeless, but an amusing side activity.
Replies:
5
In going through server logs, I sometimes see requests for /etc/passwd in the web server logs. I guess I could give them something to chew on. 🤡
@tragiccommons Obligatory graybeard story. Before the web, when we shared files across the internet via anonymous ftp, the research.att.com server would happily let you fetch the ostensible /etc/passwd file, which had a short list of uses with encrypted passwords that, when cracked, spelled out a snide message.
@mattblaze @tragiccommons Yup—Ches and I told the story in Chapter 1 of “Firewalls”; see https://wilyhacker.com/1e/chap01.pdf
@tragiccommons This was, as I recall, Bill Cheswick's mischief.
