[OpenAFS] Running OpenAFS on top of GPFS?

[Hartmut Reuter]

One of the big advantages of GPFS is that it is a fast cluster filesystem. With
normal AFS you can not make use of this feature because the fileserver doesn't
share his partitions with anyone else. So the fileserver would use GPFS just as
his local fileystem instead of XFS or something else. GPFS is good for large
files while many many files in AFS are small. In our cell with 800 TB and 200
million files 92 % of the files are smaller than 1 MB which is a reasonable
block size for GPFS.

However, since some years a special version of AFS called AFS/OSD exists which
allows to store large files in object storage. This object storage are servers
running a program called rxosd. The idea is to keep the small files in the
fileserver's partition where the volume resides and have the large files in
object storage. Now the point why GPFS is of special interest here: the GPFS
used by the rxosd could be shared by all the compute nodes in a cluster and the
modified AFS client would allow users on these compute nodes to read and write
data form and to the AFS files located inside the GPFS rxosd partition directly
with nearly the native GPFS speed (200-300 MB/s depending on the network being
used). User's outside the cluster would see the files as normal AFS files and
access them with the normal low transfer rate of AFS. I gave a talk about this
some years ago:

"Embedded Filesystems (Direct Client Access to Vice Partitions)" Talk at AFS &
Kerberos Best Practice Workshop 2009, Stanford, 2009, which you can download
from "http://www.rzg.mpg.de/~hwr/Stanford.pdf"

If you wan't to know more about this, feel free to contact me.

Hartmut Reuter

Craig Strachan wrote:
> Dear All,
> 
> The Central Computing Service at Edinburgh University is introducing a new
> University wide filesystem intended for research based data. We in
> Informatics have been asked about the possibility of us using of some of this
> new file space to either expand our existing cell or (more likely) set up a
> new cell for the whole University to use. Unfortunately, this new research
> file system is based on GPFS and so this would involve us running AFS on top
> of GPFS.
> 
> Does anyone on this list have experience of running AFS on top of GPFS which
> they would be willing to share with us? Failing that, would anyone like to
> make an educated guess as to the problems we are likely to encounter if we
> try this?
> 
> Any advice would be appreciated,
> 
> Craig. --- Craig Strachan, Computing Officer, School of Informatics,
> University of Edinburgh
> 
> 
> 
> 
> 


-- 
-----------------------------------------------------------------
Hartmut Reuter                  e-mail 		reuter@rzg.mpg.de
			   	phone 		 +49-89-3299-1328
			   	fax   		 +49-89-3299-1301
RZG (Rechenzentrum Garching)   	web    http://www.rzg.mpg.de/~hwr
Computing Center of the Max-Planck-Gesellschaft (MPG) and the
Institut fuer Plasmaphysik (IPP)
-----------------------------------------------------------------