[OpenAFS] Fwd: Unify NAS file serving & policy based tiering

[Russ Allbery]

John Rudd <jrudd@ucsc.edu> writes:

> One of the things I remember hearing from AFS people is about how it
> handles an abstraction layer between the name space and the actual
> location of the storage.  It sounds like this product is basically
> providing that for groups of NFS servers.

> I don't know if they're going to be supporting NFSv4, but if they do, I
> wonder what the main differences are that would remain between AFS and
> NFS (with this appliance, and kerberos auth in v4).

You can throw up arbitrary systems as AFS servers and don't have to buy
specialized hardware.  AFS is free.  AFS has been doing this for over a
decade and is extremely well-tested and incredibly stable, so you don't
have to worry about bugs in experimental products or new protocols.  AFS
supports more platforms than NFSv4 by quite a large margin at present
(although over time this will change, I'm sure).  AFS also has the concept
of volume replication; I don't know whether NFSv4 also has that.

Of course, there's also the flipside:  NFSv4 supports Kerberos v5 in some
ways better than AFS does.  NFSv4 doesn't use AFS ACLs, which is either a
plus or a minus depending on how you look at it.  NFSv4 has more industry
mindshare, particularly in the storage appliance sector, than AFS does.

I wrote up the following rather extended list of what we use AFS for and
what any file system would have to do to replace it for our own internal
ponderings a while back.  Here it is, in case anyone finds it interesting.
NFSv4 does a bunch of this, but not all of it, so far as I've seen.  Not
mentioned in the below is, of course, the question of migration costs for
a site that's been running AFS for a long time.

 * The file system must be truly distributed and location-independent so
   that we can migrate sets of files between servers without users
   noticing.  Some of this need would be removed by true storage
   virtualization underneath the file system servers, but we still need
   to evacuate failing hardware and upgrade systems or appliances
   periodically.  We leverage this feature extensively.

 * Strong authentication is a requirement.  UID-based authentication is
   not sufficent (the document agrees with this).  Furthermore, at least
   some degree (preferrably a high degree) of network encryption of the
   data is required.  This strong authentication must integrate with the
   campus Kerberos infrastructure.

 * File system infrastructure (basic directory trees and the like),
   critical data like the university home page, and shared software must
   be replicated geographically for fault tolerance and business
   continuence, with client failover to a different replica in the event
   of an outage.  Similarly, the server or mechanism that drives this
   replication must itself be replicated.

 * The distributed file system preferrably should provide a preview
   feature and a release operation to allow easy previewing of content
   changes and then atomic release of a set of changes.  This could
   alternatively be provided at the application level, but we make
   extensive use of this as a file system feature and duplicating it at
   the application level would be development that would have to be
   included in the project cost.

 * Clients for the *same* single distributed file system must be available
   for all supported Unix and desktop operating systems (Windows, Mac OS
   X, Linux, Solaris, IRIX, AIX, HP-UX, and Digital Unix at the current
   time, unless the decision is made to drop support for one or more of
   these platforms), or alternately the same file system with the same
   features must be exportable in some fashion that is recognized on all
   of those platforms.  Solaris, Linux, Windows, and Mac OS X are the
   obvious first-tier platforms.

 * The file system must support ACLs with more granularity than Unix
   permissions; in particular, we make extensive use of the ability to
   grant permissions to create files but not modify or delete them, to
   only read files, to traverse directories but not read the files in
   them, and to read and write to files but not change the permissions.
   These permissions must be based on the strongly-authenticated client
   identities.

 * The file system must have some mechanism for user-defined arbitrary
   groups that can be placed on ACL and that can be managed by end-users
   all over campus so that ACL management doesn't require any central
   intervention.

 * The file system must be accessible from off-campus to an appropriately
   authenticated user.

 * Either the file system must have support for redirection of paths based
   on the architecture of the client or it must support some alternative
   method of multi-platform software provisioning with minimal client-side
   configuration.

 * All administrative functions on the distributed file system must be
   fully scriptable via non-interactive command-line processes and
   automated monitoring tools.  A GUI must never be required to manage any
   routine operation of the file system (including allocating,
   reorganizing, or freeing storage, manipulating ACLs, users, and groups,
   monitoring, restarting servers, and generating reports).

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>