[OpenAFS] Extremely poor write performance.
Nathan Neulinger
nneul@umr.edu
16 Jan 2003 18:54:36 -0600
Another data point for the performance discussion...
I just happened to test this again today, and noticed stellar
performance for my test fileserver machine. The performance numbers
earlier today were around 5 MB/sec Read, 3.8 MB/sec write to a
completely idle server.
I restarted the server to test a new cvs build of the file server.
Numbers are now 11.1/10.5/etc. on read and write to that same server.
Client has 1000BT, server has 100BT, both on same switch.
That makes me wonder if there is a leak/hash issue/etc. in the
fileserver that a restart may clear up. It's possible that it may have
been a loading issue on the client, but the server has been completely
idle both times.
The fileserver I am testing against is running 2.4.20-pre7 on a
P4-1.7Ghz, with 512MB, with reiserfs based filesystem on a 3ware mirror.
Here's the full run... I'm almost tempted to have afsfsperf run
repeatedly continuously through the night to see if there is any change
in the numbers in the morning.
Note, on a 100BT client, the write performance drops maybe .1MB/sec, and
the read performance drops dramatically, to 7.7MB/sec. So there is
definately some issue there with the networking, although that dramatic
drop is a little surprising to me.
Operation total time summary
gettime 1 times : 0 msec 1418.440 operations/s
gettime 100 times : 15 msec 6327.512 operations/s
gettime 1000 times : 205 msec 4856.632 operations/s
gettime 2000 times : 327 msec 6109.370 operations/s
writing 1M data : 91 msec 10.963 Mb/s
reading 1M data : 90 msec 11.097 Mb/s
removing 1M data : 2 msec 395.257 operations/s
writing 2M data : 180 msec 11.066 Mb/s
reading 2M data : 179 msec 11.134 Mb/s
removing 2M data : 4 msec 486.027 operations/s
writing 10M data : 896 msec 11.152 Mb/s
reading 10M data : 895 msec 11.167 Mb/s
removing 10M data : 15 msec 650.407 operations/s
writing 20M data : 2022 msec 9.890 Mb/s
reading 20M data : 1791 msec 11.166 Mb/s
removing 20M data : 31 msec 641.643 operations/s
writing 30M data : 2690 msec 11.151 Mb/s
reading 30M data : 2795 msec 10.732 Mb/s
removing 30M data : 47 msec 632.178 operations/s
writing 40M data : 3656 msec 10.939 Mb/s
reading 40M data : 3815 msec 10.484 Mb/s
removing 40M data : 62 msec 637.125 operations/s
writing 50M data : 4494 msec 11.125 Mb/s
reading 50M data : 4554 msec 10.978 Mb/s
removing 50M data : 78 msec 640.246 operations/s
creating 1000 files : 896 msec 1115.479 operations/s
removing 1000 files : 790 msec 1265.387 operations/s
creating 2000 files : 1675 msec 1193.455 operations/s
removing 2000 files : 1853 msec 1079.267 operations/s
creating 4000 files : 3873 msec 1032.620 operations/s
removing 4000 files : 3414 msec 1171.539 operations/s
creating 8000 files : 7477 msec 1069.889 operations/s
removing 8000 files : 7396 msec 1081.531 operations/s
creating 1000 dirs : 1062 msec 941.603 operations/s
removing 1000 dirs : 956 msec 1045.155 operations/s
creating 2000 dirs : 2184 msec 915.491 operations/s
removing 2000 dirs : 2206 msec 906.345 operations/s
creating 4000 dirs : 4807 msec 832.020 operations/s
removing 4000 dirs : 4160 msec 961.422 operations/s
creating 8000 dirs : 10571 msec 756.740 operations/s
removing 8000 dirs : 8899 msec 898.897 operations/s
done
For comparison, here is the performance from the same machine:
Operation total time summary
gettime 1 times : 10 msec 94.616 operations/s
gettime 100 times : 13 msec 7689.942 operations/s
gettime 1000 times : 119 msec 8347.245 operations/s
gettime 2000 times : 244 msec 8189.036 operations/s
writing 1M data : 73 msec 13.652 Mb/s
reading 1M data : 49 msec 20.018 Mb/s
removing 1M data : 2 msec 400.320 operations/s
writing 2M data : 127 msec 15.647 Mb/s
reading 2M data : 98 msec 20.394 Mb/s
removing 2M data : 4 msec 482.044 operations/s
writing 10M data : 640 msec 15.622 Mb/s
reading 10M data : 492 msec 20.313 Mb/s
removing 10M data : 15 msec 640.943 operations/s
writing 20M data : 1295 msec 15.437 Mb/s
reading 20M data : 1054 msec 18.960 Mb/s
removing 20M data : 31 msec 641.601 operations/s
writing 30M data : 1932 msec 15.521 Mb/s
reading 30M data : 1468 msec 20.432 Mb/s
removing 30M data : 48 msec 618.098 operations/s
writing 40M data : 2587 msec 15.458 Mb/s
reading 40M data : 1952 msec 20.488 Mb/s
removing 40M data : 61 msec 647.857 operations/s
writing 50M data : 3232 msec 15.468 Mb/s
reading 50M data : 2540 msec 19.679 Mb/s
removing 50M data : 78 msec 633.979 operations/s
creating 1000 files : 849 msec 1177.349 operations/s
removing 1000 files : 829 msec 1205.617 operations/s
creating 2000 files : 1702 msec 1174.845 operations/s
removing 2000 files : 1663 msec 1202.590 operations/s
creating 4000 files : 3498 msec 1143.396 operations/s
removing 4000 files : 3522 msec 1135.715 operations/s
creating 8000 files : 7364 msec 1086.311 operations/s
removing 8000 files : 7582 msec 1055.008 operations/s
creating 1000 dirs : 1076 msec 928.533 operations/s
removing 1000 dirs : 983 msec 1017.073 operations/s
creating 2000 dirs : 2142 msec 933.389 operations/s
removing 2000 dirs : 2125 msec 940.914 operations/s
creating 4000 dirs : 4834 msec 827.353 operations/s
removing 4000 dirs : 4205 msec 951.201 operations/s
creating 8000 dirs : 10580 msec 756.111 operations/s
removing 8000 dirs : 8903 msec 898.532 operations/s
done
Other notes - there appears to be a very slight performance increase for
the non-pthreads fileserver, but not really enough to quantify reliably.
None of the fileserver tuning options affected above numbers much at
all.
I'm going to do that mass run against the file server and see if there
are any differences in the morning.
-- Nathan
------------------------------------------------------------
Nathan Neulinger EMail: nneul@umr.edu
University of Missouri - Rolla Phone: (573) 341-4841
Computing Services Fax: (573) 341-4216