ℹ️ (increased) scraping ℹ️
Since roughly a week there is increased scraping taking place on mountains.social. This results in higher load on the server (which you might have noticed) and also an increase in media storage (as older posts are read more frequently, they are no candidate anymore for daily housekeeping).
I remove media for remote posts older than 4 days (they will be fetched again when someone needs that media again). The used media storage was stable around 90 GB, but has increased to 200 GB in the past week. Of course this will increase storage costs.
As a first measure I have changed the housekeeping to delete media from remote posts older than 3 days instead of 4. In the coming days I will have to setup blocking IP addresses / ranges from the seen scrapers, as the currently configured robots.txt ("please do no scrape") is just plainly ignored by them.
Will not write down what I think about them, but I guess you can figure (hint: it is quite explicit).
