[OpenAFS] Afs User volume servers in VM's

[Stephan Wiesand]

On Oct 26, 2011, at 18:02 , Jeffrey Altman wrote:

> On 10/26/2011 10:34 AM, Booker Bense wrote:
>>=20
>> I am sure I'm far from the first person to think of this and there =
are
>> some threads on the list about it. But has anyone
>> gone to the logical conclusion for user volumes and done
>> one VM , one server per user home volume ?
>=20
> This is not a practical use of resources.  There is a limit of ~255 =
file
> servers in a cell which will not cover all of the users in most cells.

Thanks, I wasn't aware of that limit - which would be quite relevant if =
we were serving all our data using AFS.

> Fixing this requires a database upgrade and RPC updates for the
> management tools.
>=20
>> A batch system of any reasonable size is pretty much a built in
>> denial of service attack for the current OpenAFS implementation.
>=20
> Before deciding on a solution, what needs to be understood is which
> resource or resources become the bottleneck.  Only then can a proper
> solution be implemented.  We know there are a variety of bottlenecks =
in
> the AFS file server:
>=20
> 1. rx related

probably

> 2. file server host management

possibly

> 3. callback processing

possibly

> 4. vol package
>=20
> 5. dir package

can't comment on those due to lack of insight <:-)

> 6. disk i/o channel

unlikely

> 7. mp scalability or lack thereof

probably

> 8. other
>=20
> which are affected by the configuration of the file server and the
> system as a whole.

[...]

> Hosting multiple file server VMs on a machine is a practical way of
> improving utilization of systems with large numbers of CPU cores.  =
There
> is no benefit in deploying shipping OpenAFS file servers on machines
> with more than four cores.

It has become really hard (impossible?) to purchase a decent server with =
no more than four cores... servers with twelve cores are the current =
sweet spot if the software can handle them, and next year it will be =
sixteen cores or more.

I wonder why 128 threads (or 256 with 1.6) can't make use of more than =
four cores though?

I asked this question at EAKCS11, but let me ask once more: Would it =
make sense to host, say, three AFS fileservers on a current 12-core (two =
socket westmere) system?


--=20
Stephan Wiesand
DESY -DV-
Platanenenallee 6
15738 Zeuthen, Germany