[OpenAFS] Advice on using BTRFS for vicep partitions on Linux

[Ciprian Craciun]

On Wed, Mar 22, 2023 at 10:30=E2=80=AFAM <spacefrogg-openafs@spacefrogg.net=
> wrote:
> OpenAFS implements its own CoW and using CoW below that again has no bene=
fits and disturbs the fileservers "free-space" assumptions. It knows when i=
t makes in-place updates and does not expect to run out of space in that si=
tuation.


At what level does OpenAFS implement CoW?  Is it implemented at
whole-file-level, i.e. changing a file that is part of a replicated /
backup volume it will copy the entire file, or is it implemented at
some range or smaller granularity level (i.e. it will change only that
range, but share the rest)?

I'm asking this because I've assumed (based on empirical observations)
that all files stored in OpenAFS (via the proper `afsd`) will end
somewhere in `/vicepX` as individual files.  (I.e. if I were to
`md5sum` all the files from `/afs/some-cell`, and then `md5sum` all
the files in `/vicepX`, then the first set of `/afs/...` is a subset
of the second one `/vicepX`.)



> > Unfortunately (at least for my use-case) losing the checksumming and
> > compression is a no-go, because these were exactly the features that
> > made BTRFS appealing versus Ext4.
>
> If you say so...
> AFS does its own data checksumming.


Can one force OpenAFS to do a verification of these checksums and
report back any issues?

What kind of checksums are these?  Cryptographic ones like
MD/SHA/newer or CRC-ones?



> > Also, regarding RAID scrubbing, it doesn't cover the issue of
> > checksumming, because (for example with RAID5) it can only detect that
> > one of the disks has corrupted data, but couldn't say which.
>
> Do not use RAID to prevent data loss! That's what backups are for. RAID i=
s for operative redundancy. Scrubbing also tells you about your state of FS=
 metadata. So, it's not that it has no use without checksumming. I only use=
 RAID 1 and 1-0. They have lower dataloss probabilities that RAID 5.


Granted, RAID is not a backup solution, but it should instead protect
one from faulty hardware.  Which is exactly what it doesn't do 100%,
because if one of the drive in the array returns corrupted data, the
RAID system can't say which one is it (based purely on the returned
data).  Granted, disks don't just return random data without any other
failure or symptom.

With regard to file-system scrubbing, to my knowledge, only those that
actually have checksumming can do this, which currently is either
BTRFS or ZFS.



> All -sync properties are ineffective with NAS, because the network layer =
and far-end OS decide on actual commit strategies. So you might as well sto=
p deceiving yourself and disable write barriers.

I think that barriers have other implications especially to journaled
file-systems.



> You will use subvolumes the moment you start making snapshots. So be care=
ful to not deceive yourself. A forgotten snapshot can easily get you into t=
rouble the moment you move off some volumes to make room for a large additi=
on, just to realise no space opened up at all.

This is true.  It is true even of OpenAFS backup volumes.  :)


Ciprian.