[OpenAFS] Using AFS in a cluster?

[Tim Theisen]

I am thinking about using AFS in a Rocks Cluster.

I have a job that runs in two parts.  The first part computes
intermediate data and writes it (about 1 Gig) into the local file
system.  The second part reads this intermediate data and does a final
calculation.  The intermediate data can be used lots of times for
different final calculations.  The final job will rebalance/recalculate
the intermediate data if nodes appear/disappear.  We currently use small
clusters, so all nodes participate in the calculation.

I would like to use larger clusters and provide for the possibility that
the nodes doing the final calculation are not the same nodes that did
the intermediate calculation.  However, in most cases, the set of nodes
will be the same.  So, I desire a network file system that provides
location independence with strong local caching.  I think that AFS would
fit that bill.  If the front end is the fileserver and each node has a 4
Gigabyte cache, I think that I get what I want.

Each client would store the intermediate results into AFS.  The
intermediate data stays in the local cache to be read again.  If this
node becomes unavailable, the data can be read from the fileserver.

Do you think this is a good idea?  Are there other possibilities that I
should consider?

...Tim
--
       Tim Theisen                     Lead Research Software Engineer
Phone: +1 608 824 2848                 TomoTherapy Incorporated
  Fax: +1 608 824 2996                 1240 Deming Way
  Web: http://www.tomotherapy.com      Madison, WI 53717-1954 =20