Id like to put my lab servers to work archiving US federal data thats likely to get pulled - climate and biomed data seems mostly likely. The most obvious strategy to me seems like setting up mirror torrents on academictorrents. Anyone compiling a list of at-risk data yet?

edit (2025-09-21): this became https://sciop.net and its still going, in case anyone in this thread missed it

edit (2025-09-21 pt 2): do note the date on original post, nearly a year old, we have been rolling on sciop since February or so

#sciop

I joined the #safeguardingresearch #sciop torrent swarm to keep datasets from USA science institutions easily available.

Setup a qbittorrent Docker https://docs.linuxserver.io/images/docker-qbittorrent/
Added dataset feeds by threat level (I went with extinct, takedown_issued and endangered) https://sciop.net/feeds
Done

#Sciop hit a Petabyte (actually a Pebibyte but nobody knows that word) of total proven capacity a week or two ago. That's all the seeders * the size of the things they are seeding. All volunteers, zero dollars in funding, piggybacking off existing resources wherever we can, run on a donated VPS. This is before we even get into federating archives and are still nailing down the basics of the site.

Peer to peer archives are real and they work, period. 216TiB of threatened cultural, climate, queer, and historical information held in common. That's a people powered archive, and you're welcome in it - to take from, to add to, and help sustain if you can.

#Sciop hit a Petabyte (actually a Pebibyte but nobody knows that word) of total proven capacity a week or two ago. That's all the seeders * the size of the things they are seeding. All volunteers, zero dollars in funding, piggybacking off existing resources wherever we can, run on a donated VPS. This is before we even get into federating archives and are still nailing down the basics of the site.

Peer to peer archives are real and they work, period. 216TiB of threatened cultural, climate, queer, and historical information held in common. That's a people powered archive, and you're welcome in it - to take from, to add to, and help sustain if you can.

First post on the #sciop blog - on adding webseeds from the web interface and why this is cool for bridging archives and bringing bigger systems into the swarm. If you can't run a bittorrent client but still want to seed, this post is for you!

with art from @aud and @lina

https://blog.sciop.net/2025-08-29/webseeds

First post on the #sciop blog - on adding webseeds from the web interface and why this is cool for bridging archives and bringing bigger systems into the swarm. If you can't run a bittorrent client but still want to seed, this post is for you!

with art from @aud and @lina

https://blog.sciop.net/2025-08-29/webseeds

The slides of my talk at #WHY2025 "Safeguarding Research & Culture: Save public data from the digital bookburnings!" are now online:
https://hu.berlin/SRC-WHY2025

Recording should be available in the coming days.

More context: https://program.why2025.org/why2025/talk/B8DANE/

#SafeguardingResearch#SciOp @SafeguardingResearch

Last week trump announced plans to "review" 8 Smithsonian museums. Today he doubled down, very explicit about the intent to revise history to reflect the ethno-nationalist fantasy of US history.

You can do something about that! We are backing up the digital archives of those museums on sciop: https://sciop.net/tags/smithsonian

You can take direct action to preserve the historical artifacts the right wants to destroy:

1) you can download a copy and seed it, every seeder counts. Subscribe to the smithsonian RSS feed to auto-download torrents as they are scraped.

2) we have also written a crawler connected to sciop that distributes the scraping work, and automatically creates and uploads a validated torrent that piggybacks off the s3 bucket as a webseed source while it lasts (instructions in reply).

The data from the 8 threatened museums is on the order of ~10 TB, and we have split it up by jpg/tif so people without much spare storage can join in on the jpg's at least. The full contents of the public smithsonian bucket is ~700TB, so if we want to have a full independent copy we'll need lots more seeders.

All this code is being written flat out, on the run, as it's needed by volunteers with exactly zero resources, so it's not polished or well documented, and if you're interested in helping damp the flames of the book burning by contributing to any of the code or docs, we'd love to have you.

#Smithsonian#Sciop

Last week trump announced plans to "review" 8 Smithsonian museums. Today he doubled down, very explicit about the intent to revise history to reflect the ethno-nationalist fantasy of US history.

You can do something about that! We are backing up the digital archives of those museums on sciop: https://sciop.net/tags/smithsonian

You can take direct action to preserve the historical artifacts the right wants to destroy:

1) you can download a copy and seed it, every seeder counts. Subscribe to the smithsonian RSS feed to auto-download torrents as they are scraped.

2) we have also written a crawler connected to sciop that distributes the scraping work, and automatically creates and uploads a validated torrent that piggybacks off the s3 bucket as a webseed source while it lasts (instructions in reply).

The data from the 8 threatened museums is on the order of ~10 TB, and we have split it up by jpg/tif so people without much spare storage can join in on the jpg's at least. The full contents of the public smithsonian bucket is ~700TB, so if we want to have a full independent copy we'll need lots more seeders.

All this code is being written flat out, on the run, as it's needed by volunteers with exactly zero resources, so it's not polished or well documented, and if you're interested in helping damp the flames of the book burning by contributing to any of the code or docs, we'd love to have you.

#Smithsonian#Sciop