[OpenAFS] Re: [ Openafs : cache on zfs ]

[milek@task.gda.pl]

> -----Original Message-----
> From: openafs-info-admin@openafs.org [mailto:openafs-info-
> admin@openafs.org] On Behalf Of Andrew Deason
> Sent: 03 October 2013 19:17
> To: openafs-info@openafs.org
> Subject: [OpenAFS] Re: [ Openafs : cache on zfs ]
> 
> On Thu, 3 Oct 2013 19:34:27 +0200
> nicolas prochazka <prochazka.nicolas@gmail.com> wrote:
> 
> > Hello again ,
> > after some tests to use zfs as afs cache, linux kernel tells :
> > BUG : soft lockup - CPU0 stuck for 23s ! [ afs_cachetrim:2908]
> >
> > Any ideas are welcome,
> 
> It seems pretty likely from your other message that using zfsonlinux
> for the openafs client cache is not going to work at all until someone
> takes the time to add support for it. Just use another filesystem.
> 
> Even on other platforms, ZFS has some characteristics that make it not
> ideal for a cache, and in the past I've recommended using something
> else when it's easy to do so (e.g. UFS on Solaris, even if it's jus on
> a zvol). At least on Solaris, ZFS does some somewhat unique things with
> space allocations that have created some semi-unavoidable problems for
> the cache manager.
> 
> I can understand if someone is using ZFS for their root fs, and they
> don't want to make a separate fs just for the cache, but if you're
> making an fs just for the cache or something, it doesn't make a lot of
> sense.


We've been running AFS cache on ZFS directly for some time now, with no
issues.
The trick is to set AFS cache size somewhat smaller than the amount of space
present by the file system (so essentially we slightly over-provision). On
top of it we enable ZFS compression on the cache file system, so in reality
we overprovision even more. All works fine. 

Given that AFS cache tends to be small (several GBs, anyone is using
considerably larger cache?), a small over-provisioning is not an issue.

For example, create ZFS file system for AFS cache as:

# zfs create -o atime=off -o quota=8G -o reservation=8G -o compression=lzjb
-o recordsize=8K -o mountpoint=/afscache rpool/afscache

And in cacheinfo the specified cache size is: 6710886

This is less than 2GB overprovisioned (assuming nothing will compress), but
with modern disk sizes of OS it doesn't matter at all and seems to be a safe
enough margin. The compression is not for disk space saving it is more for
doing less i/o in some cases (depending on data).


--
Robert Milkowski
http://milek.blogspot.com