@rwg I had blocks in the reverse proxy for hard coded user agent strings and IP ranges first. This soon did not help anymore. Presently I have deployed an anubis proof-of-work container between reverse proxy and backend service and it was really easy (I am no using any special config). I now have a tenth of the scraping requests in comparison with before, i.e. two per minute make it through to the backend whereas before it was about 20 (really very rough estimate). If scraping traffic increases again, I'll try and see how easy it is to set up iocaine.
But this all is for a backend that is not very robust and tolerant of many parallel requests. It sounds like your system, by contrast, can take a fair bit of scraping traffic. Then, I would either just not do anything or consider iocaine if I felt I just wanted to push back a bit as a matter of principle.
