Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
Soblow Xaselgio :dragn_heart:
Soblow Xaselgio :dragn_heart:
@Soblow@eldritch.cafe  ·  activity timestamp 7 hours ago

If you self-host services on the internet, you may have seen waves of crawlers hammering your websites without mercy.

To annoy them and protect my services from DDoS, I decided to setup an iocaine instance, along with NSoE... And it worked... Too well.

Recently, they started flooding my VPS so much it started choking.
If you followed me here on Fedi, you saw my journey to find a way to relieve my server.

This is a rant about LLM crawlers, and some observations & conclusions, along with some techniques to help you protect your own services.

Read it here: https://xaselgio.net/posts/26.poisoning-knowledge/

#selfHosting #iocaine #indieWeb

Indigo's den

Poisoning the knowledge

A rant about current state of the internet (LLM crawlers), and some observations & conclusions, along with some techniques to help you protect your own services.
⁂
More from
Soblow Xaselgio :dragn_heart:
  • Copy link
  • Flag this post
  • Block
A Vengeful Mouse
A Vengeful Mouse
@algernon@come-from.mad-scientist.club replied  ·  activity timestamp 6 hours ago

@Soblow Thank you for writing this, reading it has been very educational!

I have a number of things in development that will make iocaine less of a pain, and more suitable for cases like yours - but... that takes a bit of time.

And NSoE's documentation is a mess, indeed. My excuse is that it was never meant to be used by anyone else, it's something I wrote for me. But there was no other option for a long time, and even if iocaine3 has a built-in script now, that's not as good as NSoE (yet).

I have plans to address that shortcoming, so there's an option that isn't NSoE, has useful, navigatable documentation that isn't written like a mad scientist's diary1.

But all the issues you listed are valid, and you even highlighted shortcomings I wasn't aware of, and tricks I did not consider. Now I have more things to play with! flan_cheer


  1. That was highly amusing to read, and I chuckled. Thanks!2 ↩︎

  2. Yes, I know, it's not a praise, but... algernon looks at the domain he's tooting from... yeah. ↩︎

  • Copy link
  • Flag this comment
  • Block

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.2-alpha.27 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Members
  • Code of Conduct