[OpenAFS] how?: Distribute /home to n Terminal Servers

[Nathan Davis]

Well, if you use at least three db + file servers, you can at least get 
a more robust system that a single NFS server can offer.  Sure, if one 
of the servers goes down you lose access to all home volumes on it. 
 But, you can distribute the volumes across the servers so at least most 
users can access their home directories if a single server goes down. 
 Also, in theory, you should be able to use fail-over with AFS servers 
just as easily and reliably as with NFS servers.

AFS also offers several benefits over NFS, including a richer 
authorization model and higher security.  And, of course, you can move 
volumes around with a single command, so its easy to keep your loads 
balanced and expand as necessary.

About "manageability":  I personally don't think AFS is much (if any) 
more complicated than NFS.  Sure, if things do wrong, it's not fun. 
 But, I haven't had any more trouble (so far) with AFS than I've had 
with my experience with NFS (which, granted, isn't much for either one). 
 The only thing I can think of that makes AFS administration a bit of a 
hassle is creating volumes, but then that pays off with more flexibility 
in the long-run.

--Nathan Davis

Dan Pritts wrote:

>AFS won't do what you want it to do. If you went with AFS, you would
>still essentially have a single point of failure for any given user's
>home directory.
>
>While AFS does give you a unified namespace with multiple file
>servers, that does not mean that any given piece of data can necessarily,
>at the backend, live in more than one place.
>
>For an AFS volume (a volume is a logical unit of storage, such as a a
>user's home directory or the directory for a particular software package)
>to be replicated across multiple AFS file servers, that volume needs to
>be set read-only.  This is obviously unworkable for a home directory.
>
>So, each read-write volume lives on a particular server, and if that
>server is down, the user is hosed. 
>
>a situation like you describe would be doable, and would take you away
>from the situation of "one NFS server dies, all users are dead", but
>would leave you with a significantly harder to manage cluster, 
>and would still leave you with "any one server dies, some set of users
>are dead".  
>
>If it were me, I would probably just stick with one NFS server.  The cost
>of the extra hardware is not that high, and you will more than make it
>up in manageability.  Obviously, it would be worth your while for any
>sizeable installation to give some effort/$$ to making sure that that
>NFS server did not go down.  You might invest in a hardware RAID with
>multiple host SCSI controllers, and use some sort of clustering software
>to ensure that if the primary NFS server died, that a secondary machine
>would take over.  
>
>Such hardware raids are not too terribly expensive - I just bought a
>system with 8x80G IDE disks (host attachment via Ultra160 scsi), dual
>PSU, single controller (but i am pretty sure that dual-controller configs
>are available for a few hundred more) for under US$5000.
>
>
>  
>
>>We are looking at an alternative because ;
>>
>>a.  A backend /home NFS Server is a single point of failure, and it is 
>>not easy to have this 'mount' failover in case this machine faults.
>>
>>b.  The NFS Server is an unnecessarily 'extra' box, when the same 
>>functionality could be provided by a distributed filesystem between 
>>existing Terminal Servers.
>>
>>c.  It is simpler.  The installed, complete system will be distributed 
>>as a ISO Set, so there will be very few AFS configuration for the 
>>sysadmin to deal with.
>>
>>The load-balancer will direct a login attempt to any one of the desktop 
>>servers and users would expect to have their account act as normal, with 
>>all their files intact.
>>
>>At the next login attempt, the same user may end up logged-in to an 
>>entirely different machine, with different DFS cache contents, but we 
>>need the  /home/<user>/* contents to be ready-synced for them to use.
>>
>>I have examined all sorts of solutions over the last two months, 
>>including `rsync`ing /home/<user>/* to all other machines on "user 
>>logout", Intermezzo DFS, CODA DFS, and now I wish to test and examine a 
>>solution with OpenAFS.
>>
>>Please do tell me right away if I am entirely mis-directed.  8-)
>>
>>
>>    
>>
>>>*every* machine that wants to access files that live in /afs needs to
>>>be an afs client.  This includes AFS server machines. 
>>>
>>>      
>>>
>>ok.  Since we have a small number of machines, what would be the 
>>situation running *all* (max 10) machines as AFS Servers, and then 
>>acting as their own clients,
>>
>>    
>>
>>>For your configuration, I woudl recommend a single AFS file & db server,
>>>which may or may not also be one of your XDMCP servers (i would tend
>>>to have it on a separate machine).  
>>>
>>>      
>>>
>>ok.  Will the clients still sync if the server is down ?  Will we still 
>>have our single point of failure ?  Can more, or all of the Terminal 
>>Servers act as OpenAFS Servers, as well as a client for the cell ?  Can 
>>we then `mount` this cell as /home ?
>>
>>    
>>
>>>Depending on what it is that you are trying to get by using AFS (eg,
>>>access across multiple sites/client side caching, or security, or
>>>what), you may find that NFS is sufficient for your needs.  
>>>
>>>      
>>>
>>We simply want 2-10 Terminal/Application Servers that keep /home/* + 
>>UIDs + GIDs sync'ed at all times.  All these Terminal/App Servers are on 
>>the same physical Ethernet Segment, firewalled in, on one site, and 
>>probably in the same rack - although it would be nice to distribute 
>>across the complex/campus near their associated set of Terminals.
>>
>>    
>>
>>>You will definitely find that NFS is easier to set up and administer.
>>>
>>>      
>>>
>>Oh Yes.  I agree.  But that functionality is limiting us now, and we 
>>wish to move on.
>>
>>
>>
>>Any and all comments would be most welcome.
>>
>>
>>TIA,
>>Steve
>>
>>
>>_______________________________________________
>>OpenAFS-info mailing list
>>OpenAFS-info@openafs.org
>>https://lists.openafs.org/mailman/listinfo/openafs-info
>>    
>>
>
>
>danno
>--
>dan pritts                                       danno@internet2.edu
>systems administrator                            734/352-4953 office
>internet2                                        734/546-4423 mobile
>_______________________________________________
>OpenAFS-info mailing list
>OpenAFS-info@openafs.org
>https://lists.openafs.org/mailman/listinfo/openafs-info
>  
>