okay so ahum, what I did there on line 8 is correct? (And why isn't this opt-in instead of opt-out?)
Discussion
Loading...
Post
![Screenshot
# This is the default rule, which allows search engines to crawl your site (recommended).
User-agent: *
Allow: /
# If you do not want AI bots to crawl your site, remove the # from the following lines:
User-agent: AI2Bot
#User-agent: Ai2Bot-Dolma
#User-agent: Amazonbot
[list continues]](https://media.twoot.site/media_attachments/files/116/053/439/692/625/279/original/b2b5672a8ea6245e.png)
I mean, this is a bit confusing. I removed the # signs. Should I have moved the whole list to the disallow section? That sounds like it makes a lot of sense, but I'm just following the instructions.
@bebatjof The last line is the instruction that will disallow the agents listed above it. Remove the # character to activate it.
See for example https://robotstxt.com/ai