[OpenAFS] cache performance

[Phil.Moore@morganstanley.com]

>>>>> "Nathan" == Nathan Neulinger <nneul@umr.edu> writes:

Nathan> What sort of additional logging are you looking for in the
Nathan> file server?

Nathan> Also, how do you plan on handling the "if it's already in the
Nathan> cache, the file server probably won't see a request" issue?

Nathan> Adding more logging is relatively easy to do, just come up
Nathan> with a list.

Well, let me first outline what information we glean from the client
audits we do today.

The goal of our client cache audits is to obtain, for each host, a
list of the volumes accessed by that client in the last 24 hours, as
well as the amount of data (not so important, but interesting).  By
collecting this information for all clients in a given cell, I can
then determine for each AFS volume in my cell, the list of clients
that have accessed that volume in the last 24 hours.

Today, by auditing the volume headers cell-wide[*], I can determine the
lastaccess timestamp, but not the list of clients accessing it.   

Now, what I would like to get on the fileserver is additional logging
which allows me to generate this kind of report on a daily basis.
Additional log entries in the FileLog are probably sufficient, but I
would be concerned about the volume of information added to the log,
as well as the logfiles volatility.

For example, the default behavior is to save *one* FileLog after a
restart.  Perform more than one restart in a single day, and you lose
some logging information.  Actually, the more I think about it, the
less I like using the FileLog.  We really need a mechanism for
maintaining these statistics that is more robust.

I'm also worried about the volume of information that would be
generated into the FileLog, which should be the place to look for
problem diagnostics, which you don't want to hide among a cloud of
performance/usage statistics.

I think a better mechanism would be to have the fileserver maintain
this information in memory, and to query it (and reset it)
periodically, via a new RPC.  Then, I can ask for this data as often
as I want, once a day, or maybe once an hour, if I need to do finer
grained analysis.

Phil

[*] Bonus question: anyone know where I'm getting these timestamps?
I'll bet you can't guess...