[OpenAFS] need help tuning openafs

[Wes Chow]

I've been migrating our system over to AFS and I've been having a lot
of difficulty tuning it for our use.  (previous we had a combination
of NFS and custom software that crudely approximated AFS -- reminds me
of how Paul Graham says that any sufficiently advanced programming
language is an approximation of Lisp!)

We have several TB of data in large (several megabyte to tens of
megabytes) files stored on AFS servers which need to be accessed by a
farm of client machines. The usage pattern for clients is that they
tend to access about 30 gigs of the same data repeatedly.  That set
changes slowly over time, but for the most part it's pretty constant.
On the server side, several gigs of data get appended daily, but data
already present rarely changes.

I figured that AFS was perfect for this type of usage.

Each client machine has a 45 gig cache.  We're starting to get
machines in that have 150 gigs of client cache.  We're running Debian
Sarge with OpenAFS 1.2.11.

I'm playing with a few of the 45 gig machines now and I've noticed
that afsd startup times are unacceptably long.  I took the advice from
the AFSLore wiki and used "afsd -chunk 17", however just starting afsd
on a clean /var/cache/openafs directory takes 20-30 minutes.  On an
already populated cache directory, it takes maybe 3 or 4 minutes.
After it's done, afs_cachetrim runs 80-90% cpu for an unkown period of
time.  (I have yet to see one finish, and it's been close to 2 hours).

My questions are:

am I passing appropriate parameters in to afsd?

what does afs_cachetrim do, and is there anything I can do to speed it
up?

is use of ext3 recommended over ext2?  (fyi, I tried with reiserfs
before reading the AFSLore wiki and it kernel panicked fairly often)

Any pointers would be greatly appreciated.  Thanks!

Wes