[OpenAFS] Afs User volume servers in VM's

Stephan Wiesand stephan.wiesand@desy.de
Wed, 26 Oct 2011 18:22:13 +0200

On Oct 26, 2011, at 18:02 , Jeffrey Altman wrote:

> On 10/26/2011 10:34 AM, Booker Bense wrote:
>> I am sure I'm far from the first person to think of this and there =
>> some threads on the list about it. But has anyone
>> gone to the logical conclusion for user volumes and done
>> one VM , one server per user home volume ?
> This is not a practical use of resources.  There is a limit of ~255 =
> servers in a cell which will not cover all of the users in most cells.

Thanks, I wasn't aware of that limit - which would be quite relevant if =
we were serving all our data using AFS.

> Fixing this requires a database upgrade and RPC updates for the
> management tools.
>> A batch system of any reasonable size is pretty much a built in
>> denial of service attack for the current OpenAFS implementation.
> Before deciding on a solution, what needs to be understood is which
> resource or resources become the bottleneck.  Only then can a proper
> solution be implemented.  We know there are a variety of bottlenecks =
> the AFS file server:
> 1. rx related


> 2. file server host management


> 3. callback processing


> 4. vol package
> 5. dir package

can't comment on those due to lack of insight <:-)

> 6. disk i/o channel


> 7. mp scalability or lack thereof


> 8. other
> which are affected by the configuration of the file server and the
> system as a whole.


> Hosting multiple file server VMs on a machine is a practical way of
> improving utilization of systems with large numbers of CPU cores.  =
> is no benefit in deploying shipping OpenAFS file servers on machines
> with more than four cores.

It has become really hard (impossible?) to purchase a decent server with =
no more than four cores... servers with twelve cores are the current =
sweet spot if the software can handle them, and next year it will be =
sixteen cores or more.

I wonder why 128 threads (or 256 with 1.6) can't make use of more than =
four cores though?

I asked this question at EAKCS11, but let me ask once more: Would it =
make sense to host, say, three AFS fileservers on a current 12-core (two =
socket westmere) system?

Stephan Wiesand
Platanenenallee 6
15738 Zeuthen, Germany