[OpenAFS] Re: Client Cache Question
Timothy Balcer
timothy@telmate.com
Tue, 9 Jul 2013 16:17:44 -0700
--001a11c2b80e531c3e04e11c5e6a
Content-Type: text/plain; charset=ISO-8859-1
FYI folks, after some offline discussion and clarification on interesting
developments in AFS, I've regressed this and I am bulk transferring to an
rsync module pointed right on top of the AFS client running on the
fileserver itself.
This gives me almost rsync native speeds for bulk transfers to AFS. It
breaks security, of course. But for performance purposes, and on a private
VLAN, it serves my purpose well.
On Fri, Jun 28, 2013 at 4:15 PM, Andrew Deason <adeason@sinenomine.net>wrote:
> On Fri, 28 Jun 2013 12:29:38 -0700
> Timothy Balcer <timothy@telmate.com> wrote:
>
> > On another VM in the same local colo, I am seeing an order of
> > magnitude (20 - 35 minutes vs 2-3 minutes) longer transfers of even
> > large single files (900M file in this case, on a cache of 500M).
> > Strace on a simple 'cat' redirect from a local disk into AFS shows
> > blocking on a write:
>
> I'm not sure if I'm following this entirely; you're getting an order of
> magnitude slower when cat'ing to /afs compared to... scp?
>
> I thought before you were just trying to see why performance was varying
> so wildly and due to different parameters; if you're just asking why the
> client is performing so slowly compared to non-AFS methods, from the
> numbers above I don't think there's any configuration you can do you
> change that.
>
> If you have a consistent RTT of 64ms, the maximum theoretical throughput
> you'll get from an AFS transfer with the code you're running is less
> than 700 KiB/s if I did my math right (32 * ~1400bytes / .064s).
> Additionally, any individual new file will take at least 128-192ms each
> (1 RTT for creating the file, 1 RTT for sending data, and possibly 1
> more RTT for changing file status). Note that those are theoretical
> limits; that is, if the client and server run infinitely fast. From the
> numbers above, it looks like they fall within the normal range.
>
> AFS network communication is currently very heavily affected by network
> latency. Anything with TCP is going to appear much faster beacuse the
> TCP window sizes will be huge, relatively speaking. Any performance
> benchmarking with UDP is going to appear much faster because it doesn't
> need to keep track of data windows, or it uses a different protocol on
> top of UDP that uses a much larger window.
>
> Does that clear up any confusion you may have had through all of this?
> AFS' poor high-latency network performance has been known for a while;
> there are / have been a few efforts to improve it, one of which I'm
> working on right now. I can talk more about that, but if this stuff
> isn't what you were asking about, then nevermind :)
>
> --
> Andrew Deason
> adeason@sinenomine.net
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
--
Timothy Balcer / IT Services
Telmate / San Francisco, CA
Direct / (415) 300-4313
Customer Service / (800) 205-5510
--001a11c2b80e531c3e04e11c5e6a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>FYI folks,=A0 after some offline discussion and clari=
fication on interesting developments in AFS, I've regressed this and I =
am bulk transferring to an rsync module pointed right on top of the AFS cli=
ent running on the fileserver itself.<br>
<br></div>This gives me almost rsync native speeds for bulk transfers to AF=
S. It breaks security, of course. But for performance purposes, and on a pr=
ivate VLAN, it serves my purpose well.<br></div><div class=3D"gmail_extra">
<br><br><div class=3D"gmail_quote">On Fri, Jun 28, 2013 at 4:15 PM, Andrew =
Deason <span dir=3D"ltr"><<a href=3D"mailto:adeason@sinenomine.net" targ=
et=3D"_blank">adeason@sinenomine.net</a>></span> wrote:<br><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;=
padding-left:1ex">
<div class=3D"im">On Fri, 28 Jun 2013 12:29:38 -0700<br>
Timothy Balcer <<a href=3D"mailto:timothy@telmate.com">timothy@telmate.c=
om</a>> wrote:<br>
<br>
> On another VM in the same local colo, I am seeing an order of<br>
> magnitude (20 - 35 minutes vs 2-3 minutes) longer transfers of even<br=
>
> large single files (900M file in this case, on a cache of 500M).<br>
> Strace on a simple 'cat' redirect from a local disk into AFS s=
hows<br>
> blocking on a write:<br>
<br>
</div>I'm not sure if I'm following this entirely; you're getti=
ng an order of<br>
magnitude slower when cat'ing to /afs compared to... scp?<br>
<br>
I thought before you were just trying to see why performance was varying<br=
>
so wildly and due to different parameters; if you're just asking why th=
e<br>
client is performing so slowly compared to non-AFS methods, from the<br>
numbers above I don't think there's any configuration you can do yo=
u<br>
change that.<br>
<br>
If you have a consistent RTT of 64ms, the maximum theoretical throughput<br=
>
you'll get from an AFS transfer with the code you're running is les=
s<br>
than 700 KiB/s if I did my math right (32 * ~1400bytes / .064s).<br>
Additionally, any individual new file will take at least 128-192ms each<br>
(1 RTT for creating the file, 1 RTT for sending data, and possibly 1<br>
more RTT for changing file status). Note that those are theoretical<br>
limits; that is, if the client and server run infinitely fast. From the<br>
numbers above, it looks like they fall within the normal range.<br>
<br>
AFS network communication is currently very heavily affected by network<br>
latency. Anything with TCP is going to appear much faster beacuse the<br>
TCP window sizes will be huge, relatively speaking. Any performance<br>
benchmarking with UDP is going to appear much faster because it doesn't=
<br>
need to keep track of data windows, or it uses a different protocol on<br>
top of UDP that uses a much larger window.<br>
<br>
Does that clear up any confusion you may have had through all of this?<br>
AFS' poor high-latency network performance has been known for a while;<=
br>
there are / have been a few efforts to improve it, one of which I'm<br>
working on right now. I can talk more about that, but if this stuff<br>
isn't what you were asking about, then nevermind :)<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
--<br>
Andrew Deason<br>
<a href=3D"mailto:adeason@sinenomine.net">adeason@sinenomine.net</a><br>
<br>
_______________________________________________<br>
OpenAFS-info mailing list<br>
<a href=3D"mailto:OpenAFS-info@openafs.org">OpenAFS-info@openafs.org</a><br=
>
<a href=3D"https://lists.openafs.org/mailman/listinfo/openafs-info" target=
=3D"_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a><br>
</div></div></blockquote></div><br><br clear=3D"all"><br>-- <br><span style=
=3D"border-collapse:collapse;color:rgb(102,102,102);font-family:verdana,san=
s-serif;font-size:x-small">Timothy Balcer / IT Services<br>Telmate / San Fr=
ancisco, CA<br>
Direct / </span><span style=3D"border-collapse:collapse;font-family:verdana=
,sans-serif;font-size:x-small"><font color=3D"#1155cc">(415) 300-4313</font=
><br><font color=3D"#666666">Customer Service /=A0</font><a value=3D"+18002=
055510" style=3D"color:rgb(17,85,204)">(800) 205-5510</a></span>
</div>
--001a11c2b80e531c3e04e11c5e6a--