Post · bonfire.cafe

Post

datacentre project status so far for those who give a shit:

- agreed to pricing and terms from interactive (42ru rack, 4kw power & 1x cross-connect) & aussie broadband (1gbit DIA & /28 subnet) for a total of $2626/m inc. GST, been waiting all week for them to send me a contract to sign, silence since monday arvo

- will have 30RU & 2kw spare for a while, worked out I can sub-lease this out to anyone interested for about 60c per watt per month (e.g: if your server uses 150W it's $90/m inc GST) with unlimited fair use of the 1gbit link, your own IP address, OOB/console access via tailscale and your own access card to the rack/DC

- waiting for CPUs & RAM to arrive from overseas to fill up one of the supermicro servers, ETA for CPU is 5th March, ETA for RAM is monday

- waiting to hear back from various aussie supermicro partners if they can get me an NVMe kit for my server, going to keep scouring EU & US sites in the meantime

- mucking around with virtfusion instead of proxmox for the BYO HDD idea, seems great for what i need, hope it doesn't throw any surprises at me while I'm testing it

- catching up next week with a former colleauge who is the smartest network engineer I know to get some tips & sanity check my network design so I don't fuck it up when people start paying me (i've been off the tools for a while!)

- pretty much decided on an Arista 7050TX-64 as a switch (48x 10gbps RJ45 ports & 4x 40gbps QSFP+ ports) and OPNsense as a router, I still need to decide on a cheap, but power efficient 1RU box to that has at least two 10gbps SFP ports to run it on

- building a website & community forum, along with some documentation

- eta for starting to move gear into the DC is april 13th

Jonathan Hedley

@jhy@tilde.zone · 3 hours ago

@decryption for the sublet, how would you measure the consumed power? Or would you go off reported peak? Interested in how that effectively works and is safe for your margin

decryption

@decryption@aus.social · 3 hours ago

@jhy pretty sure the PDUs in the rack measure it and report back - but I’d just reserve a portion for your gear, eg: you tell me you need 100W, you get 100W all month. If you use a little more that’s fine but if it’s consistent and impacting others you’d need to use less power.

s0 Traingirl Era

@s0@cathode.church · 3 hours ago

@decryption interested to hear what you end up going with for router. Pro-ish type thing or enterprise thing

decryption

@decryption@aus.social · 3 hours ago

@s0 I like OPNsense, just need to get an appropriate server for it - all the stuff like Juniper, Cisco etc all@ require expensive support contracts to keep up to date - I want to like Mikrotik but always found their gear difficult to use

s0 Traingirl Era

@s0@cathode.church · 5 hours ago

@decryption that’s a good go on that Arista. Very nice switches. If you have the cash!

decryption

@decryption@aus.social · 5 hours ago

@s0 they're EOL so not very expensive! can get one sent to Australia for under $400
https://www.bargainhardware.co.uk/arista-dcs-7050tx-64-f-48xej-45-10g-4xqsfp-40g-extensible-os-managed-switch

Bargain Hardware

Arista DCS-7050TX-64-F 48xEJ-45 10G 4xQSFP+ 40G Extensible OS Managed Switch

Part Numbers: DCS-7050TX-64-F Max Data Rate: 10Gbps Ports: RJ-45 Brand: Arista

s0 Traingirl Era

@s0@cathode.church · 5 hours ago

@decryption woah! Sick! And here I thought old Cisco 3850s were a decent deal.

decryption

@decryption@aus.social · 5 hours ago

@s0 bargainhardware.co.uk is awesome for refurb server/enterprise gear - i find them cheaper than Aussie ebay stores, they ship faster and have way more configs too

s0 Traingirl Era

@s0@cathode.church · 5 hours ago

@decryption and probably more trustworthy provenance. Twice I’ve ordered from local resellers and they’ve sent the wrong thing — “it’s a 48 port switch isn’t it?” Yeah nah mate you sent a 100meg one fucks sake

decryption

@decryption@aus.social · 5 hours ago

@s0 lol, most of them seem to be ewaste collectors that happen to find good stuff every now and then, not like, knowledgeable on the actual gear

fukawi2

@phs@aus.social · 5 hours ago

@decryption Happy to put an extra set of eyes over any configs etc for ya! 🙂

raspberry

@theraspb@aus.social · 6 hours ago

@decryption awesome to hear you doing this, happy to help with whatever time i have on anything (network, website, docs)

decryption

@decryption@aus.social · 6 hours ago

@theraspb appreciate it :)
if i run into anything will let ya know

decryption

@decryption@aus.social · 7 hours ago

most urgent thing i want to sort out is the NVMe kit for the server, these are the parts I need

BPN-NVMe4-228N-S4 - backplane

4x MCP-220-00178-0B - drive trays

CBL-PWEX-1093-26 - power cable

2x CBL-SAST-1049 - SlimSAS (SFF-8654) to SlimSAS (SFF-8654) cables

the drive trays are easy, they're on ebay but the rest is a real pain - will cost more in shipping than the parts themselves from various european vendors

Andre

@PCOWandre@jauntygoat.net · 6 hours ago

@decryption You might want to see if there's a 3d printable drive tray option.

Andre

@PCOWandre@jauntygoat.net · 6 hours ago

@decryption I have a pile of backplane cables left over from some DL380G11 upgrades; reckon they might be what you're looking for?

decryption

@decryption@aus.social · 6 hours ago

@PCOWandre hmmm, possibly, do you have a HP part number for those?

Andre

@PCOWandre@jauntygoat.net · 6 hours ago

@decryption Let's have a butchers.

decryption

@decryption@aus.social · 6 hours ago

@PCOWandre hmm, HP's description (SFFX4 P1-4 TO PRI TRI-MODX4) doesn't tell me much, haha - i think it's just an SFF-8654 to SFF-8654 cable, but I can't find a photo of the BPN-NVMe4-228N-S4 backplane to confirm!

Andre

@PCOWandre@jauntygoat.net · 6 hours ago

@decryption The cables are the same on each end, 4 to 4, and yes, SFF-8654 looks about right. I've got a box of this random so there might be something different in there.