[OpenAFS] System resources requirements and performance tuning for AFS file servers

Brian Sebby sebby@anl.gov
Thu, 14 Aug 2014 18:22:17 -0500


--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="windows-1252"

I=92m starting a project to migrate our AFS cell from the ancient =
Solaris servers that it currently lives on to a number of RHEL VMs in =
our VMware infrastructure.  One of the significant issues we=92ve had =
for a long time is that performance is lousy on our current servers, and =
I=92d like to make that better.  I have a decent amount of flexibility =
in creating the VMs, and I=92ve been reading around trying to figure out =
the best combination of system resources and server options for our AFS =
servers, and have not found a good, single document with specific =
recommendations.  There=92s a lot of stuff on various web pages, Wiki =
pages, and list postings going back over the last decade or so, but I =
was wondering if I could get some recommendations on the following =
items.  Our cell is running OpenAFS 1.6.9, not heavily loaded, and we =
have only around 100 users and maybe 600-800GB of data spread across 3 =
file servers.

For DB and file servers, what is the recommended number of CPU cores and =
memory for each?  I know at one point Jeff Altman mentioned that I =
actually wanted *fewer* cores, since there would be less cache missing =
across processors, but I assume I need more than one.   I was thinking =
of having 2-core VMs created for the 3 DB servers and 3 or 4 file =
servers, with 2 or 4 GB (DB) and 8GB (FS) of memory for each.

I=92m not too worried about tuning the DB servers, but I know that =
tuning the fileservers is key.  Right now we=92re just using the -L and =
-nojumbo flags (I realize -nojumbo is deprecated), but it seems like =
tweaking the following options are recommended.  Are these still the =
recommended values?

-udpsize 128K or 256K (Yes, I know it has to be in bytes on the actual =
servers :) )
-sendsize 128K or 256K
-vc ??? (Not sure what the best value for this is.)
-cb 1.5M

Finally, when I upgrade my DB servers, I know that the =93right=94 way =
is to shut everything down, copy over the databases, and start up the =
DBs on the new hardware, but assuming everything is at OpenAFS 1.6.9, do =
you think I=92d be safe to take down one database server at a time, =
bring up a new RHEL VM with the same IP address, start the AFS =
processes, and wait for the database to propagate to the new box?

Sadly, while I know a lot about managing AFS, I have never dug as deep =
as I should have into the AFS internals.  I was lazy and let Doug Engert =
deal with that stuff, but now he=92s retired so it=92s all up to me. :)


Thanks!

Brian

--
Brian Sebby  (sebby@anl.gov)  |  Infrastructure and Operation Services
Phone: +1 630.252.9935        |  Computing and Information Systems
Fax:   +1 630.252.4601        |  Argonne National Laboratory


--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="windows-1252"

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I=92m =
starting a project to migrate our AFS cell from the ancient Solaris =
servers that it currently lives on to a number of RHEL VMs in our VMware =
infrastructure. &nbsp;One of the significant issues we=92ve had for a =
long time is that performance is lousy on our current servers, and I=92d =
like to make that better. &nbsp;I have a decent amount of flexibility in =
creating the VMs, and I=92ve been reading around trying to figure out =
the best combination of system resources and server options for our AFS =
servers, and have not found a good, single document with specific =
recommendations. &nbsp;There=92s a lot of stuff on various web pages, =
Wiki pages, and list postings going back over the last decade or so, but =
I was wondering if I could get some recommendations on the following =
items. &nbsp;Our cell is running OpenAFS 1.6.9, not heavily loaded, and =
we have only around 100 users and maybe 600-800GB of data spread across =
3 file servers.<div><br></div><div>For DB and file servers, what is the =
recommended number of CPU cores and memory for each? &nbsp;I know at one =
point Jeff Altman mentioned that I actually wanted *fewer* cores, since =
there would be less cache missing across processors, but I assume I need =
more than one. &nbsp; I was thinking of having 2-core VMs created for =
the 3 DB servers and 3 or 4 file servers, with 2 or 4 GB (DB) and 8GB =
(FS) of memory for each.</div><div><br></div><div>I=92m not too worried =
about tuning the DB servers, but I know that tuning the fileservers is =
key. &nbsp;Right now we=92re just using the -L and -nojumbo flags (I =
realize -nojumbo is deprecated), but it seems like tweaking the =
following options are recommended. &nbsp;Are these still the recommended =
values?</div><div><br></div><div>-udpsize 128K or 256K (Yes, I know it =
has to be in bytes on the actual servers :) )</div><div>-sendsize 128K =
or 256K</div><div>-vc ??? (Not sure what the best value for this =
is.)</div><div>-cb 1.5M</div><div><br></div><div>Finally, when I upgrade =
my DB servers, I know that the =93right=94 way is to shut everything =
down, copy over the databases, and start up the DBs on the new hardware, =
but assuming everything is at OpenAFS 1.6.9, do you think I=92d be safe =
to take down one database server at a time, bring up a new RHEL VM with =
the same IP address, start the AFS processes, and wait for the database =
to propagate to the new box?</div><div><br></div><div>Sadly, while I =
know a lot about managing AFS, I have never dug as deep as I should have =
into the AFS internals. &nbsp;I was lazy and let Doug Engert deal with =
that stuff, but now he=92s retired so it=92s all up to me. =
:)<div><br></div><div><br></div><div>Thanks!</div><div><br></div><div>Bria=
n</div><div><br><div apple-content-edited=3D"true">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><div>--</div><div>Brian Sebby &nbsp;(<a =
href=3D"mailto:sebby@anl.gov">sebby@anl.gov</a>) &nbsp;| =
&nbsp;Infrastructure and Operation Services</div><div>Phone: +1 =
630.252.9935 &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;Computing and =
Information Systems</div><div>Fax: &nbsp; +1 630.252.4601 &nbsp; &nbsp; =
&nbsp; &nbsp;| &nbsp;Argonne National Laboratory</div></div>

</div>
<br></div></div></body></html>=

--Apple-Mail=_B1687E75-7BF6-486E-AAD1-CC596E4C98ED--