Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
spla
spla
@spla@mastodont.cat  ·  activity timestamp 5 months ago

Fins a 658 diferents IP del robot d'Apple (Applebot/0.1; +http://www.apple.com/go/applebot) estan llistades en el log de nginx però no intenten xafardejar ni tuts, ni enllaços ni res, només miren robots.txt a veure si tenen permís per fer-ho.

#scraping

[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1

{"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15

_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name":
"mastodont.cat", "request_time": "@0.000"}

{"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1

5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name":
"mastodont.cat", "request_time": "@0.000"}
[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1 {"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15 _7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"} {"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1 5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"}
[root@mastodont ~ # cat /var/log/nginx/json_access.log |grep "Applebot" 1 {"datetime": "11/Aug/2025:19:39:22 +0200", "IP": "17.241.227.40", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.@ (Macintosh; Intel Mac 0S X 10_15 _7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"} {"datetime": "11/Aug/2025:19:39:23 +0200", "IP": "17.241.219.192", "uri": "/robots.txt", "status": "444", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac 0S X 10_1 5_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/@.1; +http://www.apple.com/go/applebot)", "referer": "(direct)", "server_name": "mastodont.cat", "request_time": "@0.000"}
  • Copy link
  • Flag this post
  • Block

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.2-alpha.2 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Members
  • Code of Conduct