[OpenAFS] Tuning the client and server
Timothy Balcer
timothy@telmate.com
Mon, 28 Jan 2013 12:58:17 -0800
--00151758846853048504d45f89e4
Content-Type: text/plain; charset=ISO-8859-1
Hey all, and I hope your AFS journey has been fruitful :)
I've been playing about with tuning the cache manager and fileserver, and
I've gotten things working fairly well. I'm now seeing fairly good write
performance, but its spotty. I am looking at afsmonitor on the client
involved, and I am seeing a large number of vcache misses in afsmonitor
(519000 misses vs 6251 hits, at the moment). My systems are Ubuntu 12.04
and I am using the 1.6.1-1 client and server packages.
Here is some data:
fileserver: /usr/lib/openafs/fileserver -L -sendsize 512000 -b 6000 -p 128
-l 15000 -s 50000 -vc 1000 -cb 8000000 -rxpck 3600 -udpsize 12582912
-busyat 1200 -vattachpar 4
client: /sbin/afsd -stat 1500000 -dcache 9300 -daemons 6 -volumes 256
-chunksize 19 -rxpck 2000 -afsdb -dynroot -fakestat
I'm still learning the ins and outs of tuning, but in following the guide,
I increased the stat parameter to a number something like the number of
files I would be touching in my bulk data operations, volume by volume. The
file size is up to a few megabytes, but averages less than half a megabyte,
so I set the chunksize to half a megabyte. The product of dcache and
chunksize is equal to less than 95% of the cache partition size, which is 5
Gigabytes. I am not dealing with loads of volumes at once in these
operations... two to four at a time.
I am not seeing calls waiting on a thread:
Trying 10.33.10.66 (port 7001):
Free packets: 653/659, packet reclaims: 0, calls: 1583, used FDs: 64
not waiting for packets.
0 calls waiting for a thread
1 threads are idle
4 calls have waited for a thread
The fileserver has only bee "roughly" tuned. I used a set pf parameters
recommended on the list and increased the callbacks, as well as changed the
udpsize to 90% of the system's allowed size, which was changed using syctl.
My concern is that vcache misses are still high after increasing the -stat
parameter to 1.5 million. The local system has no trouble with this (plenty
of memory and processor to spare) so I could increase things quite a bit
more if needed.
I feel I am missing something.. although I do understand the basics here,
and the relation of the various parameters to activity, I am guessing that
I am missing something important re: tuning client vs. fileserver for bulk
operations. Can anyone advise?
Thanks muchly,
--
Timothy Balcer / IT Services
Telmate / San Francisco, CA
Direct / (415) 300-4313
Customer Service / (800) 205-5510
--00151758846853048504d45f89e4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hey all, and I hope your AFS journey has been fruitful :)<br><br>I've b=
een playing about with tuning the cache manager and fileserver, and I'v=
e gotten things working fairly well. I'm now seeing fairly good write p=
erformance, but its spotty. I am looking at afsmonitor on the client involv=
ed, and I am seeing a large number of vcache misses in afsmonitor (519000 m=
isses vs 6251 hits, at the moment). My systems are Ubuntu 12.04 and I am us=
ing the 1.6.1-1 client and server packages.<br>
<br>Here is some data:<br><br>fileserver: /usr/lib/openafs/fileserver -L -s=
endsize 512000 -b 6000 -p 128 -l 15000 -s 50000 -vc 1000 -cb 8000000 -rxpck=
3600 -udpsize 12582912 -busyat 1200 -vattachpar 4<br>client: /sbin/afsd -s=
tat 1500000 -dcache 9300 -daemons 6 -volumes 256 -chunksize 19 -rxpck 2000 =
-afsdb -dynroot -fakestat<br>
<br>I'm still learning the ins and outs of tuning, but in following the=
guide, I increased the stat parameter to a number something like the numbe=
r of files I would be touching in my bulk data operations, volume by volume=
. The file size is up to a few megabytes, but averages less than half a meg=
abyte, so I set the chunksize to half a megabyte. The product of dcache and=
chunksize is equal to less than 95% of the cache partition size, which is =
5 Gigabytes. I am not dealing with loads of volumes at once in these operat=
ions... two to four at a time.<br>
<br>I am not seeing calls waiting on a thread:<br><br><span style=3D"font-f=
amily:courier new,monospace">Trying 10.33.10.66 (port 7001):<br>Free packet=
s: 653/659, packet reclaims: 0, calls: 1583, used FDs: 64<br>not waiting fo=
r packets.<br>
0 calls waiting for a thread<br>1 threads are idle<br>4 calls have waited f=
or a thread</span><br><br>The fileserver has only bee "roughly" t=
uned. I used a set=A0 pf parameters recommended on the list and increased t=
he callbacks, as well as changed the udpsize to 90% of the system's all=
owed size, which was changed using syctl.<br>
<br>My concern is that vcache misses are still high after increasing the -s=
tat parameter to 1.5 million. The local system has no trouble with this (pl=
enty of memory and processor to spare) so I could increase things quite a b=
it more if needed.<br>
<br>I feel I am missing something.. although I do understand the basics her=
e, and the relation of the various parameters to activity, I am guessing th=
at I am missing something important re: tuning client vs. fileserver for bu=
lk operations. Can anyone advise?<br>
<br>Thanks muchly,<br clear=3D"all"><br>-- <br><span style=3D"border-collap=
se:collapse;color:rgb(102,102,102);font-family:verdana,sans-serif;font-size=
:x-small">Timothy Balcer / IT Services<br>Telmate / San Francisco, CA<br>Di=
rect / </span><span style=3D"border-collapse:collapse;font-family:verdana,s=
ans-serif;font-size:x-small"><font color=3D"#1155cc">(415) 300-4313</font><=
br>
<font color=3D"#666666">Customer Service /=A0</font><a value=3D"+1800205551=
0" style=3D"color:rgb(17,85,204)">(800) 205-5510</a></span>
--00151758846853048504d45f89e4--