Post · bonfire.cafe

Another data corruption, fortunately not fatal, with btrfs. Two mirrored disks that have little activity. On the same server, Proxmox 9, there is also a ZFS pool (mirrored, more active). Same type of disks.

An employee mistakenly connected an electric heater to a socket protected by the UPS, and the server rebooted brutally.

Upon reboot, one of the two btrfs disks reported:

[ 167.015266] BTRFS error (device sdd): parent transid verify failed on 873906176 wanted 998679 found 998677
[ 167.017007] BTRFS error (device sdd): parent transid verify failed on 873906176 wanted 998679 found 998677
[ 167.052517] BTRFS error (device sdd): open_ctree failed mount: /btrfs: can't read superblock on /dev/sdd.

Result: unable to mount, even in degraded mode. The only way was to disconnect sdd and mount the other disk in degraded mode.

No issues with the ZFS pool.

Needless to say, I'm now copying the data to ZFS, and before tomorrow, these two disks will be a new ZFS pool.

#RunZFS #OpenZFS #btrfs #Linux #Proxmox

release_candidate

@release_candidate@mastodon.bsd.cafe replied · 2 days ago

@stefano I have never used btrfs. But I always read horror stories about it.

All I understand about btrfs is it's eternally in beta testing phase.

h3artbl33d :openbsd: :antifa:

@h3artbl33d@exquisite.social replied · 2 days ago

@stefano

Btrfs - the name says it all. "BetterFS" but misspelled by serverside Javascript hipsters. It needs to effin burn.

I'd take FAT32 over that monstrosity any day.

It is way, way worse than the tiramisu pizza abomination.

Sorry not sorry.

Stefano Marinelli

@stefano@mastodon.bsd.cafe replied · yesterday

@h3artbl33d The tiramisu pizza is a perfect example 😆
I've used btrfs - but never for critical data. Maybe for backup servers, etc, thanks to its possibility to mix and match drives and still have redundancy.
But the price is that - more than once - it ate my data.

Mason Loring Bliss

@mason@partychickens.net replied · yesterday

@h3artbl33d @stefano Butter Fuss. I hope it becomes reliable someday, but I won't use it until it's been stable for some years. Same reason I won't use native ZFS encryption. People swear by both, and they keep losing data.

Tom

@pertho@mastodon.bsd.cafe replied · 2 days ago

@stefano ZFS 2 BTRFS 0

fedops 💙💛

@fedops@fosstodon.org replied · 2 days ago

@stefano I just run my proxmox servers with a single pool for everything and multiple datasets. Don't trust btrfs.

Joel Carnat ♑ 🤪

@joel@gts.tumfatig.net replied · 2 days ago

@stefano what would explain that a low activity fs would get wrecked this way… sounds weird; unless btrfs keeps things in memory and never go sync.

Ricardo Martín :bsdhead:

@ricardo@mastodon.bsd.cafe replied · 2 days ago

@stefano These incidents always point to a poorly designed electrical installation where servers are not isolated from other electrical devices to prevent interference, particularly issues caused by harmonics 🤷🏻

Thousands of Euros behind your electrical panel 😆
(Photo of a scary old unmaintained electrical panel) — Thousands of Euros behind your electrical panel 😆 (Photo of a scary old unmaintained electrical panel)

Stefano Marinelli

@stefano@mastodon.bsd.cafe replied · 2 days ago

@ricardo it is. The server is not in a proper room, and isn't protected in the proper way

Ricardo Martín :bsdhead:

@ricardo@mastodon.bsd.cafe replied · 2 days ago

@stefano Aye, the wise move is, before even checking the computers in an office, to take a look at the electrical panel and see whether there's a super immune RCD ... then you can brace yourself 🫣

Waitman Gobble

@hello@rumbly.net replied · 2 days ago

Similar happened. I started covering unused outlets on UPS with duct tape.

Pete Orrall

@peteorrall@mastodon.bsd.cafe replied · 2 days ago

@stefano I have very little experience with #BTRFS but from what I've read it's buggy and has issues. After reading this, it doesn't sound like it's for anything but a home lab and maybe a desktop.

My confidence is not inspired.

mkj

@mkj@social.mkj.earth replied · 2 days ago

@stefano I honestly can't believe how these sorts of issues can keep cropping up with btrfs even today. A hard system shutdown or reboot is an undesirable but certainly by no means uncommon event. And btrfs certainly has all the elements that would *enable* it to handle them gracefully in as close to a guaranteed fashion as one can reasonably get. To say nothing of *one* disk in a redundant array having bad data requiring physical intervention to get to a working state... yikes!

Antonio J. Delgado

@adelgado@social.koti.site replied · 2 days ago

@mkj @stefano I had a few unexpected shutdowns with my ZFS pools and usually it's not a problem. And one it's made of USB disks and have many times failures finding the disks (because it's a consumer motherboard and doesn't support a lot of devices), and I manage to recover always, maybe an import or a clear and the pool is back on line. I was thinking about trying btrfs in some computer, but this doesn't usually happen to me with ext4 or zfs, so maybe I won't try btrfs after all

rag. Gustavino Bevilacqua

@GustavinoBevilacqua@mastodon.cisti.org replied · 2 days ago

@stefano

Don't forget to connect the employee to the UPS, too.

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances

Bonfire social · 1.0.0 no JS en

Automatic federation enabled