wxpath – Declarative web crawling in XPath
https://github.com/rodricios/wxpath
#HackerNews #wxpath #webcrawling #XPath #technology #open-source #GitHub
Cloudflare's 2025 data reveals Google's structural advantage in AI training: Googlebot crawled 11.6% of web pages vs OpenAI's 3.6%. Publishers face an impossible choice - they can't block Google's AI crawling without losing search visibility entirely, since the same bot handles both functions. #AI #WebCrawling #DigitalRights