[OpenAFS] System resources requirements and performance tuning for AFS file servers
Brian Sebby
sebby@anl.gov
Thu, 14 Aug 2014 18:22:17 -0500
--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="windows-1252"
I=92m starting a project to migrate our AFS cell from the ancient =
Solaris servers that it currently lives on to a number of RHEL VMs in =
our VMware infrastructure. One of the significant issues we=92ve had =
for a long time is that performance is lousy on our current servers, and =
I=92d like to make that better. I have a decent amount of flexibility =
in creating the VMs, and I=92ve been reading around trying to figure out =
the best combination of system resources and server options for our AFS =
servers, and have not found a good, single document with specific =
recommendations. There=92s a lot of stuff on various web pages, Wiki =
pages, and list postings going back over the last decade or so, but I =
was wondering if I could get some recommendations on the following =
items. Our cell is running OpenAFS 1.6.9, not heavily loaded, and we =
have only around 100 users and maybe 600-800GB of data spread across 3 =
file servers.
For DB and file servers, what is the recommended number of CPU cores and =
memory for each? I know at one point Jeff Altman mentioned that I =
actually wanted *fewer* cores, since there would be less cache missing =
across processors, but I assume I need more than one. I was thinking =
of having 2-core VMs created for the 3 DB servers and 3 or 4 file =
servers, with 2 or 4 GB (DB) and 8GB (FS) of memory for each.
I=92m not too worried about tuning the DB servers, but I know that =
tuning the fileservers is key. Right now we=92re just using the -L and =
-nojumbo flags (I realize -nojumbo is deprecated), but it seems like =
tweaking the following options are recommended. Are these still the =
recommended values?
-udpsize 128K or 256K (Yes, I know it has to be in bytes on the actual =
servers :) )
-sendsize 128K or 256K
-vc ??? (Not sure what the best value for this is.)
-cb 1.5M
Finally, when I upgrade my DB servers, I know that the =93right=94 way =
is to shut everything down, copy over the databases, and start up the =
DBs on the new hardware, but assuming everything is at OpenAFS 1.6.9, do =
you think I=92d be safe to take down one database server at a time, =
bring up a new RHEL VM with the same IP address, start the AFS =
processes, and wait for the database to propagate to the new box?
Sadly, while I know a lot about managing AFS, I have never dug as deep =
as I should have into the AFS internals. I was lazy and let Doug Engert =
deal with that stuff, but now he=92s retired so it=92s all up to me. :)
Thanks!
Brian
--
Brian Sebby (sebby@anl.gov) | Infrastructure and Operation Services
Phone: +1 630.252.9935 | Computing and Information Systems
Fax: +1 630.252.4601 | Argonne National Laboratory
--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="windows-1252"
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I=92m =
starting a project to migrate our AFS cell from the ancient Solaris =
servers that it currently lives on to a number of RHEL VMs in our VMware =
infrastructure. One of the significant issues we=92ve had for a =
long time is that performance is lousy on our current servers, and I=92d =
like to make that better. I have a decent amount of flexibility in =
creating the VMs, and I=92ve been reading around trying to figure out =
the best combination of system resources and server options for our AFS =
servers, and have not found a good, single document with specific =
recommendations. There=92s a lot of stuff on various web pages, =
Wiki pages, and list postings going back over the last decade or so, but =
I was wondering if I could get some recommendations on the following =
items. Our cell is running OpenAFS 1.6.9, not heavily loaded, and =
we have only around 100 users and maybe 600-800GB of data spread across =
3 file servers.<div><br></div><div>For DB and file servers, what is the =
recommended number of CPU cores and memory for each? I know at one =
point Jeff Altman mentioned that I actually wanted *fewer* cores, since =
there would be less cache missing across processors, but I assume I need =
more than one. I was thinking of having 2-core VMs created for =
the 3 DB servers and 3 or 4 file servers, with 2 or 4 GB (DB) and 8GB =
(FS) of memory for each.</div><div><br></div><div>I=92m not too worried =
about tuning the DB servers, but I know that tuning the fileservers is =
key. Right now we=92re just using the -L and -nojumbo flags (I =
realize -nojumbo is deprecated), but it seems like tweaking the =
following options are recommended. Are these still the recommended =
values?</div><div><br></div><div>-udpsize 128K or 256K (Yes, I know it =
has to be in bytes on the actual servers :) )</div><div>-sendsize 128K =
or 256K</div><div>-vc ??? (Not sure what the best value for this =
is.)</div><div>-cb 1.5M</div><div><br></div><div>Finally, when I upgrade =
my DB servers, I know that the =93right=94 way is to shut everything =
down, copy over the databases, and start up the DBs on the new hardware, =
but assuming everything is at OpenAFS 1.6.9, do you think I=92d be safe =
to take down one database server at a time, bring up a new RHEL VM with =
the same IP address, start the AFS processes, and wait for the database =
to propagate to the new box?</div><div><br></div><div>Sadly, while I =
know a lot about managing AFS, I have never dug as deep as I should have =
into the AFS internals. I was lazy and let Doug Engert deal with =
that stuff, but now he=92s retired so it=92s all up to me. =
:)<div><br></div><div><br></div><div>Thanks!</div><div><br></div><div>Bria=
n</div><div><br><div apple-content-edited=3D"true">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><div>--</div><div>Brian Sebby (<a =
href=3D"mailto:sebby@anl.gov">sebby@anl.gov</a>) | =
Infrastructure and Operation Services</div><div>Phone: +1 =
630.252.9935 | Computing and =
Information Systems</div><div>Fax: +1 630.252.4601 =
| Argonne National Laboratory</div></div>
</div>
<br></div></div></body></html>=
--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED--