spla
@spla@mastodont.cat  ·  activity timestamp 3 weeks ago

El robot DotBot era l'únic que seguia intentant "escrapejar" mastodont.cat i dic intentant perquè l'estava bloquejant.
L'he afegit a la llista de robots que no vull que "escrapejin"...

mastodont.cat/robots.txt

...a les 10:22 i 10:23 ha xarfadejat dos tuts (sense èxit pel meu bloqueig), a les 11 el robot ja ha vist que era a robots.txt i ha parat de xafardejar. A les 11:44 ha tornat a mirar robots.txt no sé si per assegurar-se però ja no xafardeja.

{"datetime": "14/Aug/2025:10:22:38 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@joan/523289", "status": "403", "user_agent": "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000"} {"datetime": "14/Aug/2025:10:23:09 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@kim/99891381465490612", "status": "403", "user_agent": "Mozilla/5.0 (c ompatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "90.000"} {"datetime": "14/Aug/2025:11:00:20 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ""} {"datetime": "14/Aug/2025:11:44:27 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ne
{"datetime": "14/Aug/2025:10:22:38 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@joan/523289", "status": "403", "user_agent": "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000"} {"datetime": "14/Aug/2025:10:23:09 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@kim/99891381465490612", "status": "403", "user_agent": "Mozilla/5.0 (c ompatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "90.000"} {"datetime": "14/Aug/2025:11:00:20 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ""} {"datetime": "14/Aug/2025:11:44:27 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ne
{"datetime": "14/Aug/2025:10:22:38 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@joan/523289", "status": "403", "user_agent": "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000"} {"datetime": "14/Aug/2025:10:23:09 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/@kim/99891381465490612", "status": "403", "user_agent": "Mozilla/5.0 (c ompatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "90.000"} {"datetime": "14/Aug/2025:11:00:20 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ""} {"datetime": "14/Aug/2025:11:44:27 +0200", "ip": "216.244.66.250", "method": "GET", "uri": "/robots.txt", "status": "200", "bytes": "1908", "user_agent": "Mozilla/5 .0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "0.000", "u pstream_response_time": ne