also still deciding to get a rack in a datacenter or DIY it in a cheap warehouse (getting some quotes for full rack co-location), I prefer the DC as that's where servers belong, but the warehouse will be useful for storing a bunch of shit I have here and I can sublet the space to some friends to offset the costs
Discussion
Loading...
Hmm, the math really doesn't work out at all.
This will end up costing 365$/year.
For that kind of money, you can easily get a cheap nas, or a low power pc.
If you have friends that are willing to subsidise you by providing power&internet for free, the break even would be around 8-12 months. And you have full control over everything.
Even if you have to pay a nominal fee to your friend for utils, yearly running-costs would be way less than the 365$.
@newhinton if you can find a friend to do that, great! Not everyone has such a friend
@decryption I found an early mood board of this project.
@haakon I have an aesthetic
@decryption I like this idea, hopefully you’d bill annually, not daily. 😁 Getting the drive back easily (and possibly quickly) would be a factor in recovery if used for off-site backup.
@zenwheel yep annual - and yeah, if you’re in Melbourne you can pick it up same day (or maybe I’ll drop it off myself!) otherwise I’ll ship it via Express Post/courier anywhere you like just for the cost of postage
@decryption this isn't something I'd use (I have no need for it) - but I do have the question: how do you handle insurance/data loss/damage, etc?
@andyb caveat emptor - it’s clear (I hope) that you’re getting non-redundant storage that isn’t backed up.
@decryption I think you should consider that other providers offer redundancy in the pricing.
@miyuru @decryption this is a good point, i guess the counter to this is if you do want drive redundancy, you just ship another drive and connect it to the same VM and mirror, may not be possible unless you carve that out at the start though.
mind you i feel like the selling point of say backblaze (which i've personally used before) is the fact that you can start small and build up, buying the actual drive you're gonna use on this service means that you may have essentially paid for capacity that you arent using, or to get a bit more space you need to purchase another drive.
i reckon this is still a neat concept though and especially if you wanted a redundant offsite backup that needs an instant 20TB+ of storage, backblaze does cost quite a bit more.
@decryption Do you have any plans about the JBOD design?
@shlee it's just a HDD attached to a VM, so you get as much as storage as the HDD you supply
looks like there's a few of ya interested in this - some questions other have asked:
* you can pick any OS you like
* zfs snapshhot uploads are a perfect use case
* multiple HDDs are cool but they're $1/day each
* IPV6 yes once I re-familiarise myself with it
* you get SSH & full shell access - it's a $1/day VPS but you BYO HDD
* can send me a HDD pre-loaded with data so you don't have to do a big initial backup
* disk SMART info available on a private dashboard & email alerts (working on this now as I think it's vital)
i'm like 85% sure I'll do this, but the more people that express an interest the more confidence I'll have that I'll at least break even on costs!
@decryption I find your ideas intriguing and would like to subscribe to your newsletter.
@decryption If you pass-thru the raw drive, users can get their own stats.
@Tubsta @decryption iscsi each drive (when you look at me horrified like that, it’s a joke! 🤣 .. securing/maintaining that would be fun*)
@decryption mild tangent, when the grand fibre-everywhere nbn was originally announced, 1Gbps connections to SAN drives were still common, so the thought that a home connection could easily mount an iscsi/SAN-style drive remotely was quite interesting. offsite backups (real ones, not just pirate media 😋) immediately became easily viable for home users .. until it got malcolmed
getting back to that kind of capability now, though i’m stuck on 100Mbps until the owners corp grows a brain, if ever (full backups take 12-18 hours..)
@froosh fucken liberal party
@Tubsta it's tough as that can't be done per disk can it, just per controller? (one controller handles 4-8-16 etc disks)
@decryption From memory (over 10 years ago) I was doing that with VMWare ESXi. I'm sure I've done it with bhyve over the years. KVM, 'should' be able to do it.
@Tubsta i'll do some experimenting
also still deciding to get a rack in a datacenter or DIY it in a cheap warehouse (getting some quotes for full rack co-location), I prefer the DC as that's where servers belong, but the warehouse will be useful for storing a bunch of shit I have here and I can sublet the space to some friends to offset the costs
now that I’ve got some idea of the costs - I’d need roughly 100-130 HDDs/customers to break even unless there’s another activity to offset the fixed costs, hmmm. $1/day might be too cheap unless there’s lots of people that are into it. I don’t know if there’s enough word of mouth to make it viable. Could get 20 easily but after that it’ll be a challenge to break out of my circle of nerds.
@decryption @europlus You may want to check the reports of Backblaze — from the amount of HDDs they are using and their pricing model, should give you a hint of the ballbark i.t.o. range of being profitable
@decryption between you and me, I've heard rumours that renting out GPUs is slightly popular at the moment...
@cthulhu hard part is affording the GPUs lol
@decryption is your plan to use some kind of Sun Thumper / 45 drives server to cram as many HDDs into 4 ru?
@haakon I’ve been doing the sums on cost, power consumption and disk capacity - DL380 G9 is best bang for buck, they’re super cheap and can get the job done with 65W CPU. Can get more modern servers/CPUs but they cost more and don’t use that much less power over a 5yr period
@decryption How would you physically connect 100+ hard drives? There's going to be a bit of capital expense in whatever solution you use there 🙁
@phs @decryption this has me thinking too. This kind of scale would need some kind of dedicated multi-drive USB thingo. I have some ideas…
@jpm @phs @decryption i liked this design https://www.youtube.com/watch?v=vVI7atoAeoo
@phs it’s not all in the one machine! spread em across multiple cheap servers, my plan is to scrounge up old DL380 G9’s and install the lowest power CPU, then run them in low power/efficiency mode. 12x HDDs works out to around 200-240W all up.
Capital costs I’m not too concerned with as they’d be an instant asset write off and I have other income to offset it against (very much so for FY25-26)
@decryption So you need 10-12 of those boxes. Might get away with passive/minimal cooling with the right rack and room!
2x24 Port switches running active/backup, or yolo it with a single 48 port switch.
Maybe a separate 24p switch for an OOB management network?
The physical space is still a killer for costs though - DC or Warehouse.
1 more replies (not shown)
@decryption I’d happily contribute to the cost of doing it in a rack if I could shove a whole spare 2U dell in that I have sitting here 😂
@lilstevie might offer cheap co-location, could help offset the fixed costs
@decryption yeah, one thing I found was that a cage per RU is significantly cheaper than any colo available in this country, but I don’t want a whole 42U 😂
@decryption I JUST setup an offsite backup location with my own hardware, otherwise I would have been interested. maybe ill have to find another use case to use it!
@theraspb that's how I came up with the idea! set up my own and I'm like "yeah this would be easier if I could just punt someone a HDD and I dump my crap there"
@decryption yeah awesome, super cool idea.
@decryption I've seen those micro shop fronts and had a similar idea to that. Of course you don't have access to all transit providers etc, but at least you aren't paying $120-$150 ex per cross connect each month.
@Tubsta @decryption likewise.
The transit provider pricing i've been able to glean isnt that much better than what these suburban circuits cost. But you dont get the IX interconnects which would be handy if you're doing a lot of traffic with domestic networks (but you have the ISP IX interconnects so **shrugs**)
@oxyhyxo @Tubsta I was told by @haakon about nexthop's 1000/1000 for $499, that's a great deal imho
https://www.nexthop.com.au/1g-internet/
and Vocus will do
2000/2000 for about $1k/m
https://www.vocus.com.au/enterprise/internet-and-networks/vocus-internet-express
dunno exactly how much 1gbps of IP transit is in a datacentre, but once the cross connect fees are added in it's gonna be at least $1500/m I reckon?
@Tubsta yeah there's lots of nice small warehouses (150-200sqm) out west for around $20k-$30k/yr - it's tough finding out what internet access is available besides NBN though (NBN capped at 500mbit upload, which might be okay but if it scales I'd like the option to at least get to 1gbit up, the more the better)
@decryption You can do 10GbE symmetrical with EE product (minimum is 250/250 these days).
@Tubsta is NBN EE generally available anywhere FTTP is?
@decryption Seriously considering this.
@decryption Any chance of a #IPv6 /64 GUA instead of IPv4 NAT/cludge?? Is it user choice for OS? I assume disk is passed through for native access?
@Tubsta I’m an ipv6 noob but I don’t see why not! Any OS is fine (I’d ask during the signup form) and yep entire disk is attached to your VM as a block device
@decryption @Tubsta mmmm, an IPv6-only disk is mighty tempting...
@Tubsta @decryption Precisely. Might have to brush up on my zfs send from unencrypted to encrypted datasets...
@phs @decryption RAW send is your friend. Also, as your boot is 40GB on SSD, you could use geli FDE and have a zpool behind that.
1 more replies (not shown)
@decryption mmm yes this very interesting.
How do you think initial setup would work when there’s no inbound SSH? And how do you access the console in case network fucks up? CPanel or something?
@jpm you get SSH! (just won’t be on port 22) but you can also send the HDD preloaded with data so you don’t have to do a huge initial backup
console access TBA - it can be done just not sure how exactly is most practical
@decryption Will you monitor the disks for SMART errors?
@lakeswimmer yep, plan is to expose the SMART data to the customer via a dashboard and some email alerts (currently experimenting with some stuff)
@decryption Cool. I've been thinking about something similar but where I host someone's disk and they host mine and we get all commie about it and it's free.
@decryption Looks interesting. The Amazon S3 Glacier maths doesn't look right (outbound data is bigger than total price); but in any case, the "outbound data" only kicks in if you restore every piece of data on the drive, which isn't too likely.
You got me wondering what I'm paying. My S3 bill is US$10.60 a month; looks like I'm only storing 187GB though. 181GB of that is not Glacier stored, either - live-to-the-web cached image resizes. (Not bad storage after about 18 years using it. I rsync most of my filestores to it every week - non-destructive rsync which never deletes anything. Mind you, it's really very hard to find out exactly how much I have, and where.)
The "gotcha" with Glacier is the wait time for the files to be restored; otherwise, your 28TB is cheaper than $1day.
@james I just used the calculator here https://calculator.aws/
@decryption I enjoyed the rabbithole looking into my S3 bill though! It turns out that because my webserver is in Dublin, but my S3 bucket is in east-1, I am paying for much more bandwidth than I need, since traffic between different regions is paid-for. Not simple to move it, but I think it might be worth it.
(Your pricing probably therefore includes the free-tier discount).
@james AWS pricing is so confusing and complicated :(
@james @decryption By design....
@decryption is there a maximum number of drives per VPS?
@sjtrny nup, but it’s $1 a day per drive
@decryption unless it’s a massive hard drive that feels quite expensive. I can get 3-4TB cloud storage for a good chunk less than $365 year.
@decryption Quite possibly yes.