spla
@spla@mastodont.cat  ·  activity timestamp 3 weeks ago

Fins a 658 diferents IP del robot d'Apple (Applebot/0.1; +http://www.apple.com/go/applebot) estan llistades en el log de nginx però no intenten xafardejar ni tuts, ni enllaços ni res, només miren robots.txt a veure si tenen permís per fer-ho.

#scraping

[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1 {"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15 _7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"} {"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1 5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"}
[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1 {"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15 _7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"} {"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1 5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"}
[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1 {"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15 _7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"} {"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1 5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"}