[OpenAFS] Re: client statistics and metrics and general design

[Andrew Deason]

On Thu, 9 Jun 2011 13:27:11 +0200
Jakub Moscicki <jakub.moscicki@cern.ch> wrote:

> The xstat_cm_test tool reports many metrics but they are sparsely
> documented (e.g.
> http://docs.openafs.org/AdminGuide/apc.html#HDRWQ618). Could someone
> shed some light on how to best get the numbers for my use-case?

I thought the xstat_cm_test timing data is understandable enough... it
just measures the response time of various RPCs.

If you want to know the time spent in an entire vfs request, I don't
think we record any timing data for that. We do offer debug logs for
each request, though, so I think it's technically possible to get timing
data from that (you can turn this on and view it with 'fstrace'
commands). But that's probably slow.

If you wanted to add recording of timing data, the best place is
probably in afs_read, afs_write, afs_open, etc functions in the
src/afs/VNOPS/, just recording the time spent in each of those
functions. That's not quite the total amount of time in a vfs op, since
there's the platform-specific glue that comes before that, but I don't
remember any high-latency operations in that glue. If you really want to
capture the time for the _entire_ op, that's better done at the layer of
the OS anyway, or even the userspace application (i.e. measuring the
amount of time in each syscall).

On Solaris you can get this information fairly easily from dtrace. So,
you could try getting it from systemtap, depending on how well that
works these days.

> Another approach which I took was to analyse the sources. As I am new
> to AFS, I wanted to have a general idea about the design of the client
> and tried to generate call graphs with tools like cflow for the AFS
> kernel module, without convincing results yet. Is there any
> documentation on the internal of client design (linux) and possibly
> some diagrams or tools/scripts/resources to get them? Anything that
> could be useful to get a better understanding of how client internally
> works and which are the relevant available statistics?

The only documentation is what you can find in src/afs in comments or in
src/afs/DOC/. The latter is probably not really helpful with the
possible exception of src/afs/DOC/afs_rwlocks, if you want to know what
various locks are for. We can try to answer any questions you have,
though, if you ask on openafs-devel, or on jabber, irc, etc...

-- 
Andrew Deason
adeason@sinenomine.net