[OpenAFS-devel] Re: AFS ... or equivalent ...

[Robert Watson]

On Wed, 16 Jan 2008, Jeffrey Hutzelman wrote:

> --On Monday, January 14, 2008 02:23:47 PM +0000 Robert Watson 
> <rwatson@FreeBSD.org> wrote:
>
>> I'd like very much to get at least the kernel parts of an AFS client into 
>> the base system.
>
> That may well be realistic for arla, though I believe there was a period for 
> a while where the kernel/arlad interface was evolving to support features 
> like chunking.  I pay only superficial attention to arla-drinkers, so I 
> don't know what the status of any of that is; for that, you'd have to ask 
> someone who is actively involved in arla development (I believe there are 
> some such people on this list).
>
> It is unlikely ever to happen for OpenAFS, in which virtually all of the 
> cache manager code is in-kernel and most of it is cross-platform.  Trying to 
> pull the OpenAFS cache manager into the FreeBSD kernel would be equivalent 
> to forking OpenAFS; what you'd get would work and would keep up with 
> FreeBSD, but it would be unlikely to keep up with OpenAFS.

I chatted with Darrick for a while on IM yesterday (or was it the day before) 
to try and get a better understanding of the OpenAFS parts, and now that I 
know a little more, agree.  My primary experience until now has been with 
Arla, which has a very stable interface between its relatively static kernel 
module and the userspace cache manager, so the main on-going engineering for 
the kernel module is tracking changes in the FreeBSD VFS rather than tracking 
Arla changes.

> The "let's just slurp everything into the main distribution so we don't have 
> to worry about stable interfaces" approach is really poor.  It encourages 
> bad engineering practice among people maintaining the main distribution, 
> discourages innovation and extension by others, and generally doesn't scale. 
> It's far better to either attempt to maintain stable external interfaces to 
> the VFS and VM subsystems, or else admit that you don't have the resources 
> to do so given the relatively small number of external users, in which case 
> you almost certainly also don't have the resources to keep on top of updates 
> to something like OpenAFS.

Right now we maintain a relatively stable VM/VFS KPI withing a major release 
(i.e, FreeBSD 6.0 -> 6.1 -> 6.2 -> 6.3), but see fairly significant changes 
between major releases (5.x -> 6.x -> 7.x, etc).  I expect to see further 
changes in VFS for 8.x (and some of the locking-related ones have already 
started going in).

The historic problem for Arla has been that instead of tracking these VFS 
changes as they are made, they had to catch up every once in a while. Normally 
that "every once in a while" has been at the point where a FreeBSD branch is 
coming to the end of support rather than when it is new and shiny.  The result 
has been that Arla is pretty hard to use with FreeBSD as you either have to 
run a relatively old version of FreeBSD, or update the Arla kernel parts 
yourself (neither exciting prospects).  In particular, if you are a FreeBSD 
kernel developer, you will never be running Arla as you are almost certainly 
running something on the development HEAD and not an aging branch.  This leads 
to a bit of a chicken-and-egg problem, in which FreeBSD developers never use 
AFS, and this almost certainly an obstacle to it getting much use in the wider 
FreeBSD community.

If there's sufficient interest in the AFS community to create and maintain a 
port of OpenAFS to FreeBSD, I think that would be wonderful.  However, in 
light of the fact that it hasn't really happened to date, I've been trying to 
think of ways to help support that community a bit better.  In the case of 
Arla, there's a quite logical path: if we import the nnpfs kernel module (but 
not cache manager), then it will track FreeBSD development and almost 
certainly work with little or no trouble on new major releases, as sweeps to 
various KPIs will happen "for free".  If that doesn't work with OpenAFS due to 
structural differences from Arla, that's a shame (because it is easy in the 
case of Arla), but life.

So let's turn the question around: to get the OpenAFS client up and running on 
FreeBSD, do you have any technical requirements not yet met by FreeBSD, or is 
it really about finding someone willing to spend some time doing the bulk of 
the technical work and track bugs for a while?

Robert N M Watson
Computer Laboratory
University of Cambridge