[OpenAFS] Improving AFS performance for Joomla and other web sites

[Steven Jenkins]

On Wed, Sep 8, 2010 at 5:32 PM, Edgecombe, Jason <jwedgeco@uncc.edu> wrote:
> Hi everyone,
>
>
>
> We currently server our entire web tree out of AFS. This includes Joomla =
and
> other PHP-based applications. While I was benchmarking our new server, I
> noticed that the following rough performance numbers:
>
>
>
> Static files served from AFS: =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ~200
> requests per second
>
> Joomla home page served out of AFS: =A0 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0 =A0=A0~30 requests per
> second
>
> Joomla home page served from local ext3: =A0=A0=A0=A0=A0=A0=A0=A0 =A01000=
 requests per second.
>
>
>
> I=92m running a 8 cores with hyperthreading (16 threads). I=92m running t=
he
> Apache threaded worker MPM on RHEL5.5 with mod_fcgid and OpenAFS 1.4.12.1=
. I
> have Apache configured for ~1000 threads.
>
>
>
> My data (250GB) and stat cache for the AFS client should be big enough to
> not have to hit the server.
>
>
>
> My afsd has the following options: =93 -fakestat -dynroot -chunksize 18 -=
stat
> 500000 -daemons 32 -volumes 2000 -blocks 212713128=94
>
>
>
> What is recommended to improve my AFS performance for Joomla?
>

Skipping past a lot of the usual caveats around performance in a
complex system and going (somewhat) straight to the AFS-specific bits:

1- It would be very useful to also compare static files served from
ext3 -- that can give you somewhat of a baseline (i.e., compare the
ratio of static files in ext3 to Joomla in ext3 versus static files in
AFS to Joomla in AFS).  Offhand, the slowdown you're seeing between
static files in AFS vs Joomla in AFS does not seem egregious (based on
my experience tuning other web application servers, but not Joomla
specifically).

2- To see if cache configuration is an issue, look at your hit/miss
rates.  You can use afsmonitor to look at that dynamically.

3- Depending on your ability to do further instrumentation, it would
be very helpful to look at traffic between your web server and AFS
servers (e.g., turn on audit logs) to see exactly what your web server
is requesting from servers.  Alternatively,  you can capture packet
traces with tcpdump/snoop/wireshark.  Don't forget your pt and
vlservers, by the way.  I personally prefer packet traces, as I'm able
to see oddities like bad addresses (and retries) that would not show
up in server logs.

Steven