[OpenAFS] Debugging a network performance problem that affects
AFS
Dale Pontius
pontius@btv.ibm.com
Fri, 14 Jan 2011 08:33:49 -0500
This is a multi-part message in MIME format.
--------------000108000805090703010804
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Now that you mention it, there *may* be a firewall involved - I'm not
sure. Some time back they began putting putting some server pools
behind firewalls, "protecting" them from the campus LAN. I don't know
if the AFS servers are one of those pools.
As for client software, this is more of a network-wide event. Many
client machines, several levels of client software, but most of the
machines are not user-installed. (mine being one of few exceptions)
I'm relatively certain that none of these machines have multiple clients
installed. Well, my machine does, because it's multi-boot and the
various OS installs have different levels of AFS client software. But
at any one time only one level of client is active.
As for cache, it's almost entirely on its own partition. The standard
install appears to allocate an 8G partition, and my own installs use a
4G partition. (The cache partition is shared between bootable OS installs.)
Dale
On 01/13/2011 03:56 PM, Anne Salemme wrote:
> hi dale,
> not knowing anything about your setup...let me tell you two things
> that took me _weeks_ to figure out at one site....might be useful for you:
>
> 1) a firewall between client and server....this one had a throttle, so
> it would only cut off if traffic got "high enough"...
>
> 2) client system had two different versions of afs client installed
> (obviously, should not have happened)...anyway, two afs clients on
> same system, fighting it out, looked like "afs is slow"
>
> one other thing to look at: the client cache setup..make sure it's not
> competing for space...
> best of luck!
>
> anne
>
> --- On *Thu, 1/13/11, Dale Pontius /<pontius@btv.ibm.com>/* wrote:
>
>
> From: Dale Pontius <pontius@btv.ibm.com>
> Subject: [OpenAFS] Debugging a network performance problem that
> affects AFS
> To: openafs-info@openafs.org
> Date: Thursday, January 13, 2011, 3:24 PM
>
> We're having intermittent network performance problems, and the
> primary manifestation to us as users looks like "an OpenAFS
> slowdown." I'm wondering if it's possible to collect access time
> statistics out of an OpenAFS Linux client. A little time with
> google and I see the "-enable_peer_stats" and
> "-enable_process_stats" options when starting the client daemon,
> and this very well may furnish the information that I need. A
> subsequent search gets me to the "rxdebug" document, though that
> document appears to be server-centric as opposed to querying the
> client. Nor does it tell me what information I can collect or if
> access time is part of that information - only mentioning serveral
> parameters that it does collect.
>
> Can someone toss me a bone here - or a link?
>
> Thanks
> Dale Pontius
>
> -- Dale Pontius
> Senior Engineer
> IBM Corporation
> Phone: (802) 769-6850
> Tie-Line: 446-6850
> email: pontius@us.ibm.com </mc/compose?to=pontius@us.ibm.com>
>
> This e-mail and its attachments, if any, may contain confidential
> and privileged material for the sole use of the intended
> recipient. Any review, use, distribution or disclosure by others
> is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the
> sender by reply e-mail and delete all copies of this message from
> your system without copying it and notify sender of the
> misdirection by reply e-mail.
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org </mc/compose?to=OpenAFS-info@openafs.org>
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
--
Dale Pontius
Senior Engineer
IBM Corporation
Phone: (802) 769-6850
Tie-Line: 446-6850
email: pontius@us.ibm.com
This e-mail and its attachments, if any, may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message from your system without copying it and notify sender of the misdirection by reply e-mail.
--------------000108000805090703010804
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Now that you mention it, there *may* be a firewall involved - I'm
not sure. Some time back they began putting putting some server
pools behind firewalls, "protecting" them from the campus LAN. I
don't know if the AFS servers are one of those pools.<br>
<br>
As for client software, this is more of a network-wide event. Many
client machines, several levels of client software, but most of the
machines are not user-installed. (mine being one of few
exceptions) I'm relatively certain that none of these machines have
multiple clients installed. Well, my machine does, because it's
multi-boot and the various OS installs have different levels of AFS
client software. But at any one time only one level of client is
active.<br>
<br>
As for cache, it's almost entirely on its own partition. The
standard install appears to allocate an 8G partition, and my own
installs use a 4G partition. (The cache partition is shared between
bootable OS installs.)<br>
<br>
Dale<br>
On 01/13/2011 03:56 PM, Anne Salemme wrote:
<blockquote cite="mid:153454.97340.qm@web305.biz.mail.mud.yahoo.com"
type="cite">
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="font: inherit;" valign="top">hi dale,<br>
not knowing anything about your setup...let me tell you
two things that took me _weeks_ to figure out at one
site....might be useful for you:<br>
<br>
1) a firewall between client and server....this one had a
throttle, so it would only cut off if traffic got "high
enough"...<br>
<br>
2) client system had two different versions of afs client
installed (obviously, should not have happened)...anyway,
two afs clients on same system, fighting it out, looked
like "afs is slow"<br>
<br>
one other thing to look at: the client cache setup..make
sure it's not competing for space...<br>
best of luck!<br>
<br>
anne<br>
<br>
--- On <b>Thu, 1/13/11, Dale Pontius <i><a class="moz-txt-link-rfc2396E" href="mailto:pontius@btv.ibm.com"><pontius@btv.ibm.com></a></i></b>
wrote:<br>
<blockquote style="border-left: 2px solid rgb(16, 16,
255); margin-left: 5px; padding-left: 5px;"><br>
From: Dale Pontius <a class="moz-txt-link-rfc2396E" href="mailto:pontius@btv.ibm.com"><pontius@btv.ibm.com></a><br>
Subject: [OpenAFS] Debugging a network performance
problem that affects AFS<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:openafs-info@openafs.org">openafs-info@openafs.org</a><br>
Date: Thursday, January 13, 2011, 3:24 PM<br>
<br>
<div class="plainMail">We're having intermittent network
performance problems, and the primary manifestation to
us as users looks like "an OpenAFS slowdown." I'm
wondering if it's possible to collect access time
statistics out of an OpenAFS Linux client. A little
time with google and I see the "-enable_peer_stats"
and "-enable_process_stats" options when starting the
client daemon, and this very well may furnish the
information that I need. A subsequent search gets me
to the "rxdebug" document, though that document
appears to be server-centric as opposed to querying
the client. Nor does it tell me what information I
can collect or if access time is part of that
information - only mentioning serveral parameters that
it does collect.<br>
<br>
Can someone toss me a bone here - or a link?<br>
<br>
Thanks<br>
Dale Pontius<br>
<br>
-- Dale Pontius<br>
Senior Engineer<br>
IBM Corporation<br>
Phone: (802) 769-6850<br>
Tie-Line: 446-6850<br>
email: <a moz-do-not-send="true"
ymailto="mailto:pontius@us.ibm.com"
href="/mc/compose?to=pontius@us.ibm.com">pontius@us.ibm.com</a><br>
<br>
This e-mail and its attachments, if any, may contain
confidential and privileged material for the sole use
of the intended recipient. Any review, use,
distribution or disclosure by others is strictly
prohibited. If you are not the intended recipient (or
authorized to receive for the recipient), please
contact the sender by reply e-mail and delete all
copies of this message from your system without
copying it and notify sender of the misdirection by
reply e-mail.<br>
<br>
_______________________________________________<br>
OpenAFS-info mailing list<br>
<a moz-do-not-send="true"
ymailto="mailto:OpenAFS-info@openafs.org"
href="/mc/compose?to=OpenAFS-info@openafs.org">OpenAFS-info@openafs.org</a><br>
<a moz-do-not-send="true"
href="https://lists.openafs.org/mailman/listinfo/openafs-info"
target="_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a><br>
</div>
</blockquote>
</td>
</tr>
</tbody>
</table>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Dale Pontius
Senior Engineer
IBM Corporation
Phone: (802) 769-6850
Tie-Line: 446-6850
email: <a class="moz-txt-link-abbreviated" href="mailto:pontius@us.ibm.com">pontius@us.ibm.com</a>
This e-mail and its attachments, if any, may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message from your system without copying it and notify sender of the misdirection by reply e-mail.
</pre>
</body>
</html>
--------------000108000805090703010804--