[OpenAFS] Afs User volume servers in VM's

Jeffrey Altman jaltman@secure-endpoints.com
Wed, 26 Oct 2011 12:45:07 -0400

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 10/26/2011 12:22 PM, Stephan Wiesand wrote:
>> Hosting multiple file server VMs on a machine is a practical way of
>> improving utilization of systems with large numbers of CPU cores.  The=
>> is no benefit in deploying shipping OpenAFS file servers on machines
>> with more than four cores.
> It has become really hard (impossible?) to purchase a decent server wit=
h no more than four cores... servers with twelve cores are the current sw=
eet spot if the software can handle them, and next year it will be sixtee=
n cores or more.

12 cores is not a sweet spot if you can't use them.

A typical server has an option of two sockets and a processor per socket
with either 4 or 6 core with or without hyper threading.  For OpenAFS
today you want to use a single socket and disable hyper threading.

> I wonder why 128 threads (or 256 with 1.6) can't make use of more than =
four cores though?

Resource contention between threads executing on separate sockets is
extremely expensive and there is no practical method of enforcing
processor affinity for all of the file server threads to ensure that
they only execute on cores within a single socket.  At least not that I
am aware of on Linux.  Windows would make such thread execution
restrictions trivial.

> I asked this question at EAKCS11, but let me ask once more: Would it ma=
ke sense to host, say, three AFS fileservers on a current 12-core (two so=
cket westmere) system?

It is certainly an approach that can be taken.  It does reduce
contention between the file server processes.  However, virtualization
will not ensure that the file server threads of any given file server
will execute on cores within a single socket.

Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

Version: GnuPG v1.4.9 (MingW32)