[OpenAFS] Heavy performance loss on gigabit ethernet

[Enrico Scholz]

rader@ginseng.hep.wisc.edu writes:

>  > we are using OpenAFS 1.2.11 in an environment where the fileserver has a
>  > 1000Mb/s ethernet interface and the clients 100Mb/s ones. With this
>  > setup we get really poor client-performance on large files; e.g. a 40 MB
>  > sized file needs nearly 4 minutes for the transfer.
>
> You may have clients getting horriable throughput because of fast
> ethernet autonegotiation problems.  Check this with "iperf".


Thx, I did some measurements with this tool in the following
environment:

   [  machine A ]       [ 3Com 4900 ]     [machine B]
   [   1000Mbs  ]-------[  Gigabit  ]-----[ 1000Mbs ]
   [(fileserver)]       [  switch   ]
                              |
                              |
                       [ 1000Mbs port ]
                       [  3Com 4200   ]
                       [  100Mbs port ]
                              |
                              |
                        [ machine C  ]
                        [   100Mbs   ]
                        [(AFS client)]
and get:


The setup of the iperf-servers:
| [ensc@B]$ iperf -su -b 1000M   # -b does not seem to be honored in '-s' mode
| [ensc@C]$ iperf -su -b  100M


Now, the test between the both 1000Mb/s machines:
| [ensc@A]$ iperf -c B -b 1000M -ud
| [ ID] Interval       Transfer     Bandwidth
| [  5]  0.0-10.0 sec   399 MBytes   334 Mbits/sec
| [  3]  0.0-10.0 sec   409 MBytes   343 Mbits/sec  0.031 ms 182132/473950 (38%)
| [  5] Server Report:
| [  5]  0.0-10.0 sec   399 MBytes   334 Mbits/sec  0.090 ms   17/284323 (0.006%)
| [  5] Sent 284323 datagrams

Ok, that's not the full gigabit speed but it might be caused by the
system which got a CPU load of 60-70%.


Now, the test between 1000Mb/s and 100Mb/s machines:
| [ensc@A]$ iperf -c C -b 1000M -ud
| [ ID] Interval       Transfer     Bandwidth
| [  4]  0.0-10.0 sec   622 MBytes   522 Mbits/sec
| [  3]  0.0-10.0 sec   114 MBytes  95.6 Mbits/sec  0.297 ms    0/81287 (0%)
| [  4] Server Report:
| [  4]  0.0-10.2 sec   114 MBytes  93.5 Mbits/sec  15.142 ms 362232/443628 (82%)
| [  4] Sent 443628 datagrams

This is expected: server A sends with full gigabit-speed and lots of
UDP packages will be dropped as client is 100Mb/s only. Therefore, the
network itself seems to be ok.

As this test corresponds to the slow AFS performance (fileserver A sends
large file to client C), something must be wrong with AFS.




Enrico