[OpenAFS] Re: Client Cache Question
Timothy Balcer
timothy@telmate.com
Tue, 25 Jun 2013 13:49:06 -0700
--047d7b6da166041aaa04e000a943
Content-Type: text/plain; charset=ISO-8859-1
Not precisely:
I am rsyncing from some files in a directory local to the AFS client, into
the afs tree, and ALSO from that same local directory, the same files, to
an rsync module. These occur sequentially.. they don't run in parallel.
The rsync server lives on the same physical machine the AFS fileserver
lives on.
So, this means that there is no networkly or I/O reason on the destination
or source that could be the culprit particularly.
Im running an fstrace right now, but as you said, its very cryptic. Are you
suggesting I run an strace on the rsync process? Or that I attach a strace
to all the afs background daemons?
Best,
On Tue, Jun 25, 2013 at 12:14 PM, Andrew Deason <adeason@sinenomine.net>wrote:
> On Mon, 24 Jun 2013 15:58:45 -0700
> Timothy Balcer <timothy@telmate.com> wrote:
>
> > - Soon after client restart, rsync is very fast.. less than a second,
> > compared to rsync modules at 3-5 seconds
> > - Then, immediately, or after a few iterations, it slows down to 40+
> > seconds. It stays this way for the duration (days, so far. no change).
> > - Rsync times to rsync modules on the same destination host do not
> > change.
> > - The amount of data is small, as is the number of files (100k or less
> > per file, and 100 or so files each time)
> > - The files are always new. They are not maintained on AFS, they are
> > sync'd TO AFS from a standard file system. They are never there
> already.
> > - Network speeds are good
>
> So if I understand correctly, you're running rsync ./some/dir
> /afs/some/dir; that is, rsync to the local machine, into /afs (and the
> afs volume is somewhere far away). And you're comparing this to running
> 'rsync ./some/dir server::foo', where the destination server is the afs
> fileserver, or close by it? And each rsync run is for different data,
> but around the same size each time.
>
> Some kind of trace could help say what's going on. I mentioned fstrace
> before, but something that's easier to get and probably more
> interpretable by you would be just a plain strace. If you run with -T,
> you'll get the amount of time for each syscall, so you could see if
> we're hanging on some specific operation. (also probably run with -f,
> and -o /path/to/file to save the output)
>
> Or run rsync with --progress, which could maybe show if we're hanging on
> a specific file or something. I think of that as more for interactive
> use, though; if you're keeping this to a more noninteractive batch
> process, I'm not sure if that's useful at all.
>
> --
> Andrew Deason
> adeason@sinenomine.net
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
--
Timothy Balcer / IT Services
Telmate / San Francisco, CA
Direct / (415) 300-4313
Customer Service / (800) 205-5510
--047d7b6da166041aaa04e000a943
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><div><div><div>Not precisely:<br><br></div>I am rsync=
ing from some files in a directory local to the AFS client, into the afs tr=
ee, and ALSO from that same local directory, the same files, to an rsync mo=
dule. These occur sequentially.. they don't run in parallel.<br>
<br>The rsync server lives on the same physical machine the AFS fileserver =
lives on.<br><br></div>So, this means that there is no networkly or I/O rea=
son on the destination or source that could be the culprit particularly.<br=
>
<br></div>Im running an fstrace right now, but as you said, its very crypti=
c. Are you suggesting I run an strace on the rsync process? Or that I attac=
h a strace to all the afs background daemons?<br><br></div>Best,<br><div>
<div><div><br><br></div></div></div></div><div class=3D"gmail_extra"><br><b=
r><div class=3D"gmail_quote">On Tue, Jun 25, 2013 at 12:14 PM, Andrew Deaso=
n <span dir=3D"ltr"><<a href=3D"mailto:adeason@sinenomine.net" target=3D=
"_blank">adeason@sinenomine.net</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">On Mon, 24 Jun 2013 15:58:45 -0700<br>
Timothy Balcer <<a href=3D"mailto:timothy@telmate.com">timothy@telmate.c=
om</a>> wrote:<br>
<br>
> =A0 =A0- Soon after client restart, rsync is very fast.. less than a s=
econd,<br>
<div class=3D"im">> =A0 =A0compared to rsync modules at 3-5 seconds<br>
</div>> =A0 =A0- Then, immediately, or after a few iterations, it slows =
down to 40+<br>
<div class=3D"im">> =A0 =A0seconds. It stays this way for the duration (=
days, so far. no change).<br>
</div>> =A0 =A0- Rsync times to rsync modules on the same destination ho=
st do not<br>
> =A0 =A0change.<br>
> =A0 =A0- The amount of data is small, as is the number of files (100k =
or less<br>
<div class=3D"im">> =A0 =A0per file, and 100 or so files each time)<br>
</div>> =A0 =A0- The files are always new. They are not maintained on AF=
S, they are<br>
<div class=3D"im">> =A0 =A0sync'd TO AFS from a standard file system=
. They are never there already.<br>
</div>> =A0 =A0- Network speeds are good<br>
<br>
So if I understand correctly, you're running rsync ./some/dir<br>
/afs/some/dir; that is, rsync to the local machine, into /afs (and the<br>
afs volume is somewhere far away). And you're comparing this to running=
<br>
'rsync ./some/dir server::foo', where the destination server is the=
afs<br>
fileserver, or close by it? And each rsync run is for different data,<br>
but around the same size each time.<br>
<br>
Some kind of trace could help say what's going on. I mentioned fstrace<=
br>
before, but something that's easier to get and probably more<br>
interpretable by you would be just a plain strace. If you run with -T,<br>
you'll get the amount of time for each syscall, so you could see if<br>
we're hanging on some specific operation. (also probably run with -f,<b=
r>
and -o /path/to/file to save the output)<br>
<br>
Or run rsync with --progress, which could maybe show if we're hanging o=
n<br>
a specific file or something. I think of that as more for interactive<br>
use, though; if you're keeping this to a more noninteractive batch<br>
process, I'm not sure if that's useful at all.<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
--<br>
Andrew Deason<br>
<a href=3D"mailto:adeason@sinenomine.net">adeason@sinenomine.net</a><br>
<br>
_______________________________________________<br>
OpenAFS-info mailing list<br>
<a href=3D"mailto:OpenAFS-info@openafs.org">OpenAFS-info@openafs.org</a><br=
>
<a href=3D"https://lists.openafs.org/mailman/listinfo/openafs-info" target=
=3D"_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a><br>
</div></div></blockquote></div><br><br clear=3D"all"><br>-- <br><span style=
=3D"border-collapse:collapse;color:rgb(102,102,102);font-family:verdana,san=
s-serif;font-size:x-small">Timothy Balcer / IT Services<br>Telmate / San Fr=
ancisco, CA<br>
Direct / </span><span style=3D"border-collapse:collapse;font-family:verdana=
,sans-serif;font-size:x-small"><font color=3D"#1155cc">(415) 300-4313</font=
><br><font color=3D"#666666">Customer Service /=A0</font><a value=3D"+18002=
055510" style=3D"color:rgb(17,85,204)">(800) 205-5510</a></span>
</div>
--047d7b6da166041aaa04e000a943--