[OpenAFS] Advice on using BTRFS for vicep partitions on Linux

spacefrogg-openafs@spacefrogg.net spacefrogg-openafs@spacefrogg.net
Wed, 22 Mar 2023 09:29:52 +0100 (GMT+01:00)


> What is the reason behind disabling copy-on-write for BTRFS?=C2=A0 Does i=
t
> break OpenAFS in some way, or is it only the out-of-space issue?

OpenAFS implements its own CoW and using CoW below that again has no benefi=
ts and disturbs the fileservers "free-space" assumptions. It knows when it =
makes in-place updates and does not expect to run out of space in that situ=
ation.

For context, AFS did most of the cool FS stuff that the young kids are doin=
g nowadays before it was cool. So, the tricky thing with other CoW filesyst=
ems is that they might work against each other.

Anyways, AFS deals badly with out-of-space situations, because it needs spa=
ce to move off a volume to make space, like any other CoW filesystem. It ha=
s it's mechanisms to prevent that from happening but CoW FS counteract thes=
e mechanisms.

>
> Unfortunately (at least for my use-case) losing the checksumming and
> compression is a no-go, because these were exactly the features that
> made BTRFS appealing versus Ext4.

If you say so...
AFS does its own data checksumming.
Compression is problematic on nearly full disks, because you can never prop=
erly judge whether the data will fit after compression or not. The gains we=
re not worth it for me.

> Also, regarding RAID scrubbing, it doesn't cover the issue of
> checksumming, because (for example with RAID5) it can only detect that
> one of the disks has corrupted data, but couldn't say which.

Do not use RAID to prevent data loss! That's what backups are for. RAID is =
for operative redundancy. Scrubbing also tells you about your state of FS m=
etadata. So, it's not that it has no use without checksumming. I only use R=
AID 1 and 1-0. They have lower dataloss probabilities that RAID 5.

> Could you elaborate more on this?=C2=A0 I guess it doesn't apply to
> directly attached disks.=C2=A0 Is this in order to increase write
> performance, or?

All -sync properties are ineffective with NAS, because the network layer an=
d far-end OS decide on actual commit strategies. So you might as well stop =
deceiving yourself and disable write barriers.

>
> Have you also changed the `-sync` file-server option?
>
> I'm using `-sync onclose` to be sure that my data is actually stored
> on the disk.=C2=A0 The write performance does suffer, especially for
> use-cases like Git where some simple operations (like repacking) take
> forever (because for some reason Git tries to touch each and every
> `.git/objects/XX` folders...)

True, I left it on 'onclose' as well IIRC.

> (In my case I intend to use a dedicated BTRFS disk, over RAID, without
> any subsolumes.)

You will use subvolumes the moment you start making snapshots. So be carefu=
l to not deceive yourself. A forgotten snapshot can easily get you into tro=
uble the moment you move off some volumes to make room for a large addition=
, just to realise no space opened up at all.

Kind regards,
=E2=80=93Michael