[OpenAFS-devel] Fileserver as Masters Thesis

[Kyle Moffett]

On Sep 19, 2005, at 09:51:13, Niklas Edmundsson wrote:
> Hi all!
>
> Me and a colleague are planning on finally getting our Master  
> Thesis done, and our current plan is to implement a new OpenAFS  
> fileserver since that would be a rather useful result of 20 weeks  
> work.

Oooh! Looks like fun!  Thanks for volunteering!

> We envision not touching the RX stuff, hoping that rxtcp will solve  
> all the various issues that's in there.

If possible, you might try initially targeting Linux and using the in- 
kernel Rx implementation.  Unfortunaty, it currently has no  
SOCK_RXRPC userspace interface, but from some initial tinkering I did  
in that area, it wouldn't be hard to bridge the kernel code to  
userspace.  If you're interested in following down that path, I can  
probably dig up the few initial patches I did and send you the  
details David Howells and I hashed out.

> Ultimately one would want the performance of the INODE fileserver  
> combined with the portability of the NAMEI fileserver. With  
> performance we're talking both small-file tinkering and scaling up  
> to next generation network speeds and filesizes.

If possible, it would be nice to have a fileserver that stored its  
ACLs in POSIX extended attributes on an existing filesystem, through  
a combination of POSIX ACLs and AFS-specific xattrs.  On the other  
hand I suspect that this, like the kernel-Rx support I mention above,  
is a Linux-only feature, or is at least restricted to Linux, one or  
two of the BSDs, maybe Mac OS X, and possibly Windows.  I'm obviously  
somewhat Linux-biased, but there are several Linux features that  
would make an extremely efficient, modern, and completely-in- 
userspace OpenAFS server practical:

   --- In-kernel Rx layer ---
This makes the whole jumbo-gram issue discussed previously a non- 
issue.  The kernel knows the constraints of the hardware and can  
repackage a 1MB hunk of data into nice normal TCP and/or UDP packet  
sizes according to the protocol using efficient zero-copy IO.

   --- Raw filesystem use and POSIX ACLs and xattrs ---
Sometimes I would really like to just make random filesystem  
directories accessible in AFS without having to put whole partitions  
in AFS in a format not directly useable by userspace tools (must run  
OpenAFS server and client to access the files).  If you could use the  
underlying filesystem directly for file storage, you could take  
advantage of all of the speed and efficiency of said underlying  
filesystem.  While that may not be so awesome on some OSes *cough*  
Windows *cough*, normal file access on those OSes isn't so great  
either, and it would let the fileserver take advantage of better  
readahead and FS cache tuning in the kernel (IE: An OpenAFS file maps  
1:1 to a file that the kernel sees).  Getting the POSIX ACL <=> AFS  
ACL mapping right would be tricky, so you should probably start with  
specially-named files or simple POSIX xattrs.  One other note about  
raw filesystem access is that it would make it impossible to migrate  
such a volume between servers with vos commands.  You could create  
special OpenAFS automatic volume trees in /afsvol[0-9]+ that are  
indexed in a database and automagically managed by the AFS server,  
including migration support, and specify that any directories outside  
of that are not migrate-able, which would seem to be the simplest  
solution.

   --- Simultaneous local and remote use (Inotify support on Linux) ---
If use the underlying filesystem directly, then it would be nice to  
allow normal local access to said filesystem to interact well with  
the AFS server.  Using inotify on Linux, you could properly break  
remote callbacks when the local file is modified, and local programs  
would be able to see when files are modified by remote clients by  
subscribing to the inotify events for those files/mounts/etc.

I hope that's somewhat along the line of what you're looking for in  
input!

Cheers,
Kyle Moffett

--
I have yet to see any problem, however complicated, which, when you  
looked at it in the right way, did not become still more complicated.
   -- Poul Anderson